Consider using Truststore to expose system certificate stores

[abdnh]

#3050 patches requests using pip-system-certs to force it to not rely on certifi. I noticed this is causing problems in add-ons (e.g. AnkiHub) with some Windows users (probably with fresh OS installations) where any request fails with this:

certificate verify failed: unable to get local issuer certificate

Unfortunately passing verify=certifi.where() to requests to get the old behavior doesn't work with the pip-system-certs patch in place. I had to switch to httpx in one add-on to work around the issue.

Truststore offers a clean solution but we have to wait until we upgrade to Python 3.10.

[dae]

If it's affecting add-ons, I imagine it's affecting things like pasting of images into the editor as well.

I wonder if the issue is a recent regression? https://gitlab.com/alelec/pip-system-certs/-/issues/27
Reverting requests is presumably not an option since we risk reintroducing previous CVEs.

Do you have a sense of how widespread the issue is? Wondering which group is more affected: those using custom certs from e.g. school/work, or those who are getting this error.

[dae]

Re Python 3.10, I'd like to see us move off PyOxidizer this year, as it's already bitrotting, and we're too far behind Python releases at this point.

[abdnh]

If it's affecting add-ons, I imagine it's affecting things like pasting of images into the editor as well.

That's right. I just reproduced this on Windows Sandbox.

I wonder if the issue is a recent regression?

Do you have a sense of how widespread the issue is?

The issue with add-ons has been here since 24.04 was released. AnkiHub actually got a report of the issue in the first day: https://community.ankihub.net/t/trouble-logging-into-ankihub/196142

We used to get daily reports in April last year. I feel I'm seeing less reports of it, but as I'm saying that I found the most recent report is hours ago: https://community.ankihub.net/t/add-on-error/396244

On Windows, the 24.04 issue appears to affect new installations, and I suspect it goes away after installing the latest updates.
We got similar reports before the update (mostly from Mac users). Such issues were mostly solved by clearing proxy settings or trying a different network.

Wondering which group is more affected: those using custom certs from e.g. school/work, or those who are getting this error.

It does appear people who are getting this error are more affected, but I'm curious why there are not many reports of it on the forums (as far as I can tell) despite the fact that it breaks image pasting.

[dae]

So the issue goes away once the user installs Windows updates? And it affects both Windows 10 and 11 users? I'm hoping to get another rc build going isoon, so as a stop-gap solution, I might try catching this error and nudging the user to update Windows.

[dae]

@abdnh does running 'certutil -syncWithWU' in the sandbox shell fix the issue?

[abdnh]

So the issue goes away once the user installs Windows updates? And it affects both Windows 10 and 11 users?

It affects Windows 10 and 11 users. I used to instruct users to download the certificate manually and install it as a quick workaround: https://community.ankihub.net/t/trouble-logging-into-ankihub/196142/14

Running certutil -syncWithWU [directory] to download the certificates then installing them using this script works (thanks ChatGPT):

@echo off
setlocal

:: Set the directory where your .crt files are located
set "CERT_DIR=C:\path\to\your\certs"

:: Loop through all .crt files in the directory
for %%f in ("%CERT_DIR%\*.crt") do (
    echo Installing certificate: %%f
    certutil -addstore "Root" "%%f"
)

endlocal
pause

So installing updates should fix the issue.

[dae]

I played around with this for a while and got it working, but we'd need to deal with UAC prompts and systems where PS execution is disabled, so I think I'll just take the easy route and add a message for this release, and we can add a better solution down the line (or wait until we update Python)