Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

package.elm-lang.org uses expired DST Root X3 certificate #355

Open
CyBeRoni opened this issue Oct 28, 2021 · 5 comments
Open

package.elm-lang.org uses expired DST Root X3 certificate #355

CyBeRoni opened this issue Oct 28, 2021 · 5 comments

Comments

@CyBeRoni
Copy link

CyBeRoni commented Oct 28, 2021

Hi,

As of this moment, package.elm-lang.org is configured to use the Let's Encrypt default chain which includes the expired root certificate mentioned in the title. Let's Encrypt uses this to provide compatibility with Android 7 and earlier (which does not check the notAfter date on trust anchor certs), but use of this chain causes problems on many non-Android OSes.

Since Android compatibility is not a necessity for this service and this configuration causes breaks in other people's infrastructure, I suggest that Certbot be set (using --preferred-chain "ISRG Root X1") to use the short ISRG Root X1 chain instead. This should not cause issues as systems without that root in their trust stores will not accept the expired DST root either, but it will solve breakage for those of us with systems that reject the certificate chain upon encountering the expired root.

@CyBeRoni
Copy link
Author

CyBeRoni commented Oct 28, 2021

For completeness the chain as offered up by the server is as follows, the offending (signed by expired root) cert being the last one:

 0 s:/CN=package.elm-lang.org
   i:/C=US/O=Let's Encrypt/CN=R3
-----BEGIN CERTIFICATE-----
MIIFLDCCBBSgAwIBAgISAzJRJUNIp7eIzHcumZTZjdOgMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTEwMDYwMDM2MjNaFw0yMjAxMDQwMDM2MjJaMB8xHTAbBgNVBAMT
FHBhY2thZ2UuZWxtLWxhbmcub3JnMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
CgKCAQEA+f7NsARWNpZBL/5bFaGGMTwwpoSYvbtzZjJ722hRnoJ5Lv7GUu5ybWiL
1oQeEvrQRr1ykl51HlqWgy0J1Af890o+0ChmB9buo/G8K4qTrBlgXRER6Cdz7jB+
tcyoV+odNSsg+DRjYfWJl1OlkRLPRoqg/VyKVeOWoD5Kv0DYG3djKaWOp2jEVO1R
GCo05mrOPvZ75VfLohbtvxPeY6y9whYrh67ebuOJm9sT/MqFi/lmJwFkUV4Z5ZPp
qrTjpBcFWgwQcao6aB63L1eMeGqLzqFweeajMCTRqxLKwP4oR7Q/R7yVUe6QZ6cf
lKGe+CqcJcOf9skkFhrGBAmfzlbs1wIDAQABo4ICTTCCAkkwDgYDVR0PAQH/BAQD
AgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAMBgNVHRMBAf8EAjAA
MB0GA1UdDgQWBBSsltF9Ks/2qgFHRnefPwLgnevKIjAfBgNVHSMEGDAWgBQULrMX
t1hWy65QCUDmH6+dixTCxjBVBggrBgEFBQcBAQRJMEcwIQYIKwYBBQUHMAGGFWh0
dHA6Ly9yMy5vLmxlbmNyLm9yZzAiBggrBgEFBQcwAoYWaHR0cDovL3IzLmkubGVu
Y3Iub3JnLzAfBgNVHREEGDAWghRwYWNrYWdlLmVsbS1sYW5nLm9yZzBMBgNVHSAE
RTBDMAgGBmeBDAECATA3BgsrBgEEAYLfEwEBATAoMCYGCCsGAQUFBwIBFhpodHRw
Oi8vY3BzLmxldHNlbmNyeXB0Lm9yZzCCAQIGCisGAQQB1nkCBAIEgfMEgfAA7gB1
AN+lXqtogk8fbK3uuF9OPlrqzaISpGpejjsSwCBEXCpzAAABfFM+mK8AAAQDAEYw
RAIgXRd1NZ0PNEqQj4dp/7S97Hc+oCYvbXZ9vfhYzniSMQICIDQ8Kmv7yGTXZy4H
jsluyqjjTNj5GsiEg+UlZepUbbXrAHUARqVV63X6kSAwtaKJafTzfREsQXS+/Um4
havy/HD+bUcAAAF8Uz6Y2AAABAMARjBEAiBqedpGdiGbxqXqvbrCSETM2jjoKQCj
er7k/xwK28C6DwIgB2FOZpylB1pjAh9aN142az/TyglgfXUI0bMHB89+v/IwDQYJ
KoZIhvcNAQELBQADggEBAFf5lwqpIWNp46G7wuDrLeINBzyHTeAdul/QJHpeks2i
JASlleOVtoZSW/yeCJDXXn/4X6oyB5V6xuZSWVMXHBi3MqcmlIwb5Y6S7OUlBWOb
CVcgMEvJI5yxkQurjiAtzk4tuK1zuzfrp3IGejPZBLe9mwB+5LBbc4vLwVx6iGWx
nSD1zpcnPyYgWVBdGXkzHkEmZ0yas+COUx+sRYhIxZLd6filw+MO3DMnmzLI5+TA
TrcfoYo5ug5G3X1eQqlNzIupIcOpJ/c83hd9diApM6YrvAMdBf7DQiRrUvHIxhO4
SXSmtF096eTgL2x9+Wz3Fjgn3sWglHX45FAXXURdL2E=
-----END CERTIFICATE-----
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
-----BEGIN CERTIFICATE-----
MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAw
WhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3Mg
RW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cP
R5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdx
sxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8Zutm
NHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxg
Z3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG
/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMC
AYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYB
Af8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaA
FHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcw
AoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRw
Oi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQB
gt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6W
PTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wl
ikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQz
CkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BIm
lJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4
avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2
yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1O
yK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90Ids
hCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+
HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6Zv
MldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqX
nLRbwHOoq7hHwg==
-----END CERTIFICATE-----
 2 s:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
-----BEGIN CERTIFICATE-----
MIIFYDCCBEigAwIBAgIQQAF3ITfU6UK47naqPGQKtzANBgkqhkiG9w0BAQsFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTIxMDEyMDE5MTQwM1oXDTI0MDkzMDE4MTQwM1ow
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwggIiMA0GCSqGSIb3DQEB
AQUAA4ICDwAwggIKAoICAQCt6CRz9BQ385ueK1coHIe+3LffOJCMbjzmV6B493XC
ov71am72AE8o295ohmxEk7axY/0UEmu/H9LqMZshftEzPLpI9d1537O4/xLxIZpL
wYqGcWlKZmZsj348cL+tKSIG8+TA5oCu4kuPt5l+lAOf00eXfJlII1PoOK5PCm+D
LtFJV4yAdLbaL9A4jXsDcCEbdfIwPPqPrt3aY6vrFk/CjhFLfs8L6P+1dy70sntK
4EwSJQxwjQMpoOFTJOwT2e4ZvxCzSow/iaNhUd6shweU9GNx7C7ib1uYgeGJXDR5
bHbvO5BieebbpJovJsXQEOEO3tkQjhb7t/eo98flAgeYjzYIlefiN5YNNnWe+w5y
sR2bvAP5SQXYgd0FtCrWQemsAXaVCg/Y39W9Eh81LygXbNKYwagJZHduRze6zqxZ
Xmidf3LWicUGQSk+WT7dJvUkyRGnWqNMQB9GoZm1pzpRboY7nn1ypxIFeFntPlF4
FQsDj43QLwWyPntKHEtzBRL8xurgUBN8Q5N0s8p0544fAQjQMNRbcTa0B7rBMDBc
SLeCO5imfWCKoqMpgsy6vYMEG6KDA0Gh1gXxG8K28Kh8hjtGqEgqiNx2mna/H2ql
PRmP6zjzZN7IKw0KKP/32+IVQtQi0Cdd4Xn+GOdwiK1O5tmLOsbdJ1Fu/7xk9TND
TwIDAQABo4IBRjCCAUIwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYw
SwYIKwYBBQUHAQEEPzA9MDsGCCsGAQUFBzAChi9odHRwOi8vYXBwcy5pZGVudHJ1
c3QuY29tL3Jvb3RzL2RzdHJvb3RjYXgzLnA3YzAfBgNVHSMEGDAWgBTEp7Gkeyxx
+tvhS5B1/8QVYIWJEDBUBgNVHSAETTBLMAgGBmeBDAECATA/BgsrBgEEAYLfEwEB
ATAwMC4GCCsGAQUFBwIBFiJodHRwOi8vY3BzLnJvb3QteDEubGV0c2VuY3J5cHQu
b3JnMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmwuaWRlbnRydXN0LmNvbS9E
U1RST09UQ0FYM0NSTC5jcmwwHQYDVR0OBBYEFHm0WeZ7tuXkAXOACIjIGlj26Ztu
MA0GCSqGSIb3DQEBCwUAA4IBAQAKcwBslm7/DlLQrt2M51oGrS+o44+/yQoDFVDC
5WxCu2+b9LRPwkSICHXM6webFGJueN7sJ7o5XPWioW5WlHAQU7G75K/QosMrAdSW
9MUgNTP52GE24HGNtLi1qoJFlcDyqSMo59ahy2cI2qBDLKobkx/J3vWraV0T9VuG
WCLKTVXkcGdtwlfFRjlBz4pYg1htmf5X6DYO8A4jqv2Il9DjXA6USbW1FzXSLr9O
he8Y4IWS6wY7bCkjCWDcRQJMEhg76fsO3txE+FiYruq9RUWhiF1myv4Q6W+CyBFC
Dfvp7OOGAN6dEOM4+qR9sdjoSYKEBpsr6GtPAQw4dy753ec5
-----END CERTIFICATE-----```

@rlefevre
Copy link
Member

Could you provide more details about "use of this chain causes problems on many non-Android OSes" and "this configuration causes breaks in other people's infrastructure" with some sources please?

@CyBeRoni
Copy link
Author

CyBeRoni commented Oct 28, 2021

These issues are well-documented.

An announcement from LE: https://community.letsencrypt.org/t/openssl-client-compatibility-changes-for-let-s-encrypt-certificates/143816, more info: https://medium.com/geekculture/will-you-be-impacted-by-letsencrypt-dst-root-ca-x3-expiration-d54a018df257

The gist of it is that clients using older versions of TLS libraries such as OpenSSL < 1.1 do not correctly verify the chain of trust by default (and client software does not generally configure the library to do it correctly), meaning that if the expired root is present in the chain they fail the verification rather than noticing that one of the intermediate certs is also a root in and of itself.

While the most up-to-date versions of said libraries do not have this problem, use of the older libraries is still widespread and those clients will now not be able to connect to package.elm-lang.org securely until they are updated, which it is not always possible for everyone to do.

Since the reason for serving this chain with an expired root by Let's Encrypt is compatibility with a mobile OS that is not relevant to the service, use of the shorter, non-cross-signed chain is more appropriate.

@rlefevre
Copy link
Member

rlefevre commented Oct 28, 2021

Thank you. Do you have some examples of applications using package.elm-lang.org with OpenSSL < 1.1?

Also is there some evidence that using the default chain breaks more clients (using OpenSSL < 1.1) than using the "ISRG Root X1" one (breaking older Android versions)?

@CyBeRoni
Copy link
Author

CyBeRoni commented Oct 28, 2021

Well l figured it out because my CI process using an older version of NodeJS broke unexplainedly.

The new ISRG X1 root (short chain) only breaks on clients that do not have that certificate in their store. The long chain breaks on older clients that do not verify the chain correctly, even if they have the new root in their store. I don't have numbers but here's a table of breakage:

Old client, ISRG Root not in store, short chain: Breaks (untrusted, expected)
Old client, ISRG Root in store, short chain: Works
Old client, ISRG Root not in store, long chain: Breaks (expired, expected)
Old client, ISRG Root in store, long chain: Breaks (expired, unexpected)
New client, ISRG root not in store, long chain: Breaks (expired, expected, should not happen)
New client, ISRG Root in store, short chain: Works
New client, ISRG root not in store, short chain: Breaks (untrusted, expected, should not happen)
New client, ISRG Root in store, long chain: Works

In summary: clients (except Android) without the ISRG root in their store are broken regardless and will need the new root cert added to their trust store. Old clients with the cert will work with the short chain but not the long one. New clients will work with either.

The only OS that benefits from using the long chain is Android 4 through 7.1 which I maintain is completely irrelevant to this service. So changing to the short chain only breaks access by this irrelevant OS, while fixing access by relevant-but-not-updated clients.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants