Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gh-131423: Update OpenSSL data to 3.4.1 on Linux #131618

Open
wants to merge 15 commits into
base: main
Choose a base branch
from
Open

gh-131423: Update OpenSSL data to 3.4.1 on Linux #131618

wants to merge 15 commits into from
+9,432 −89

Conversation

picnixz
Copy link
Member

@picnixz picnixz commented Mar 23, 2025 •
edited
Loading

I've also updated the make_ssl_data.py script that @encukou has recently updated as well. I completed with instructions that I thought usefull for future maintainers.

📚 Documentation preview 📚: https://cpython-previews--131618.org.readthedocs.build/

picnixz added 7 commits March 20, 2025 14:34
@picnixz
update OpenSSL CI build configuration
85ec7b2
@picnixz
update OpenSSL mnemonics
40ad7ce 
Since mnemonics from 3.4.1 are different (renumbered) from 3.4.0.

To ease future updates, we assume the following:

`_ssl_data_<MAJOR><PATCH>.h` contains the latest OpenSSL data. If the
previous `_ssl_data_<MAJOR><PATCH>.h` file is incompatible with the
newest one (e.g., because some mnemonics were renamed or removed), the
old one is renamed to `_ssl_data_<MAJOR><MINOR><PATCH>.h` where <PATCH>
is the patch number it was based upon.

In this commit, OpenSSL 3.4.1 mnemonics are not compatible with OpenSSL
3.4.0 mnemonics as they were renumbered. Therefore, `_ssl_data_34.h` is
renamed to `_ssl_data_340.h` and `_ssl_data_34x.h` now contains OpenSSL
3.4.1 mnemonics.

We also refined the mnemonics that are selected, discarding those that
are mnemonics-like but should not be used as such (e.g., ERR_LIB_MASK
and ERR_LIB_OFFSET for OpenSSL 1.1.1).
@picnixz
update macOS and Windows OpenSSL configurations
b35b7fa
@picnixz
Merge remote-tracking branch 'upstream/main' into ci/update/ssl-versi…
578bb0e 
…ons-131423
@picnixz
update SBOM
296a98c
@picnixz
update SBOM
326d947
@picnixz
update configure requirements
cf7a65d
@bedevere-app bedevere-app bot mentioned this pull request Mar 23, 2025
Open
4 tasks
@picnixz picnixz force-pushed the ci/update/ssl-versions-131423 branch from 05ee142 to 5bbc702 Compare March 23, 2025 10:53
@picnixz picnixz marked this pull request as ready for review March 24, 2025 14:56
encukou
encukou reviewed Mar 24, 2025
View reviewed changes
Tools/ssl/make_ssl_data.py
Comment on lines +146 to +147
# FEAT(picnixz): in the future, we may want to also check
# the consistency of the OpenSSL files with an external tool.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you track that in an issue instead?

picnixz
picnixz commented Mar 24, 2025
View reviewed changes
picnixz
picnixz commented Mar 24, 2025
View reviewed changes
picnixz
picnixz commented Mar 24, 2025
View reviewed changes
picnixz
picnixz commented Mar 24, 2025
View reviewed changes
@picnixz picnixz marked this pull request as draft March 24, 2025 17:49
@picnixz
Copy link
Member Author

picnixz commented Mar 24, 2025

arf, I'm not on my Linux so I can't regen :< I'm leaving tomorrow morning so I'm not really sure I'll be able to commit before leaving, but otherwise, just take over the PR and regen the data!

picnixz
picnixz commented Mar 24, 2025
View reviewed changes
ned-deily
ned-deily reviewed Mar 26, 2025
View reviewed changes
Copy link
Member

@ned-deily ned-deily left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The macOS build-installer.py changes LGTM, thanks!

@picnixz picnixz force-pushed the ci/update/ssl-versions-131423 branch from 38bcd15 to 41863fb Compare March 28, 2025 13:00
@picnixz
Copy link
Member Author

picnixz commented Mar 28, 2025

I'll wait for #131804 to be merged until bumping the Linux mnemonics and CI

@picnixz picnixz marked this pull request as ready for review March 28, 2025 22:50
@picnixz picnixz requested a review from encukou March 28, 2025 22:50
@picnixz picnixz requested a review from gpshead as a code owner April 5, 2025 10:52
@picnixz picnixz changed the title gh-131423: Update OpenSSL to 3.0.16 (macOS, Windows) and 3.4.1 (Linux) gh-131423: Update OpenSSL to 3.0.16 (macOS) and 3.4.1 (Linux) Apr 5, 2025
@picnixz picnixz added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 5, 2025
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @picnixz for commit 905f1a5 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F131618%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 5, 2025
@picnixz
Copy link
Member Author

picnixz commented Apr 5, 2025

To avoid surprises, I'm running the build bots. If they pass, I'll merge this one so that we can close the other issue. I think the Windows-related failures were recently solved as well

@picnixz
Copy link
Member Author

picnixz commented Apr 5, 2025 •
edited by hugovk
Loading

The iOS failure is known (PR #132050)

@picnixz
Copy link
Member Author

picnixz commented Apr 6, 2025

I want to think about something. Mnenmonics were updated in 3.4.1 compared to 3.4.0, but that's only because I knew that they were changed. However, we're actually having a

#if (OPENSSL_VERSION_NUMBER >= 0x30100000L)
#include "_ssl_data_34.h"

So I think I'll need a way to check first that when OpenSSL mnemonics changed so that we regenerate the correct files per version.

@ned-deily
Copy link
Member

@picnixz, with the 3.14.a7, 3.13.3, and 3.12.10 releases approaching in two days, I plan to update the macOS installers for those releases to use 3.0.16. If you don't expect to be able to merge this PR before then, I can pull out the build-installer.py change into a separate PR since it has no relation to any of the other changes in this PR. (And that's why I prefer to keep changes like this separate.)

@picnixz
Copy link
Member Author

picnixz commented Apr 6, 2025

If you don't expect to be able to merge this PR before then

To be on the safe side, please do so. I don't want to block the macOS side with my interrogations. Hopefully I'll be able to merge this before the release.

@picnixz
Copy link
Member Author

picnixz commented Apr 7, 2025 •
edited
Loading

Note: 3.4.1 includes CVE patches but since we're still in alpha, we can say that those security patches are not really necessary now (in addition, they affect components that are not directly exposed by Python IIRC). So, I'll postpone this until after the release (I don't want to have surprises where a mnemonic change would be actually annoying for a user)

@picnixz picnixz changed the title gh-131423: Update OpenSSL to 3.0.16 (macOS) and 3.4.1 (Linux) gh-131423: Update OpenSSL to 3.4.1 on Linux Apr 7, 2025
@picnixz picnixz changed the title gh-131423: Update OpenSSL to 3.4.1 on Linux gh-131423: Update OpenSSL data to 3.4.1 on Linux Apr 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zooba zooba zooba left review comments

@ned-deily ned-deily ned-deily left review comments

@erlend-aasland erlend-aasland Awaiting requested review from erlend-aasland erlend-aasland is a code owner

@corona10 corona10 Awaiting requested review from corona10 corona10 is a code owner

@ezio-melotti ezio-melotti Awaiting requested review from ezio-melotti ezio-melotti is a code owner

@hugovk hugovk Awaiting requested review from hugovk hugovk is a code owner

@AA-Turner AA-Turner Awaiting requested review from AA-Turner AA-Turner is a code owner

@encukou encukou Awaiting requested review from encukou

@gpshead gpshead Awaiting requested review from gpshead gpshead is a code owner

Assignees
No one assigned
Labels
awaiting core review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@picnixz @bedevere-bot @ned-deily @encukou @zooba