Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabled android-ndk to work with Python 3.12+ #809

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

veloman-yunkan
Copy link
Collaborator

@veloman-yunkan veloman-yunkan commented Mar 5, 2025

Should fix #808

distutils was dropped from Python 3.12. Fortunately, our usage of android-ndk only relied on distutils.dir_util.copy_tree() in build/tools/make_standalone_toolchain.py which
is easy to replace with shutil.copytree(). That is done via a small patch.

Note that there are more references to distutils in the following files but it looks like our CI/CD flows aren't affected by those:

  • sources/third_party/shaderc/third_party/spirv-tools/utils/generate_registry_tables.py
  • sources/third_party/vulkan/src/scripts/update_deps.py
  • prebuilt/linux-x86_64/bin/python2.7-config
  • prebuilt/linux-x86_64/bin/python-config
  • prebuilt/linux-x86_64/bin/python2-config
  • various files under prebuilt/linux-x86_64/lib/python2.7/
  • python-packages/fastboot/setup.py
  • python-packages/adb/setup.py

@veloman-yunkan veloman-yunkan force-pushed the python3.13+_friendly_android-ndk-r21e branch from 2c96505 to e81d9c4 Compare March 5, 2025 09:05
@veloman-yunkan veloman-yunkan changed the title Enabled android-ndk to work with Python 3.13+ Enabled android-ndk to work with Python 3.12+ Mar 5, 2025
Copy link
Contributor

@kelson42 kelson42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The CI should stable, with pinned dependecies, so what has changed in our code base that has introduce this regression?
  • Why this regression has not been caught in the CI?
  • CI is not passing, I can hardly merge this PR

@kelson42 kelson42 force-pushed the python3.13+_friendly_android-ndk-r21e branch from e81d9c4 to 5684372 Compare March 8, 2025 11:06
@mgautierfr
Copy link
Member

The CI should stable, with pinned dependecies, so what has changed in our code base that has introduce this regression?
Why this regression has not been caught in the CI?

This is not (yet) a regression. As we used pinned dependencies, we are still running on ubuntu 2022 and so with a python < 3.12 So CI has not detected this.

This is a issue with recent distributions (now on dev devices, later on CI on next update)

@kelson42
Copy link
Contributor

kelson42 commented Mar 11, 2025

@mgautierfr This was working and this is not anymore, so this is a regression. What exacty has changed and who has introduced this change making rhat the CI brakes?

Does the CD has other dependencies than the CI? What has changed in the CD to lead us where we are?

@mgautierfr
Copy link
Member

@mgautierfr This was working and this is not anymore, so this is a regression.

Android part CI/CD is working : https://github.com/kiwix/kiwix-build/actions/runs/13778363920/job/38531945492

Other failing jobs are not about this (and I don't know why they are broken because you ask me to not take care)

What exacty has changed and who has introduced this change making rhat the CI brakes?

Nothing has changed and this is why android CI/CD is not broken. It will be when we will change the CI/CD to next version of Python.

The CI of this PR is broken because of the content of the PR, probably the path wrongly formatted

@veloman-yunkan veloman-yunkan force-pushed the python3.13+_friendly_android-ndk-r21e branch 3 times, most recently from d45f83f to 8d6831f Compare March 13, 2025 10:20
distutils was dropped from Python 3.12. Fortunately, our usage of
android-ndk only relied on distutils.dir_util.copy_tree() in
build/tools/make_standalone_toolchain.py which
is easy to replace with shutil.copytree(). That is done via a small
patch.

Note that there are more references to distutils in the following
files but it looks like our CI/CD flows aren't affected by those:

sources/third_party/shaderc/third_party/spirv-tools/utils/generate_registry_tables.py
sources/third_party/vulkan/src/scripts/update_deps.py
prebuilt/linux-x86_64/bin/python2.7-config
prebuilt/linux-x86_64/bin/python-config
prebuilt/linux-x86_64/bin/python2-config
various files under prebuilt/linux-x86_64/lib/python2.7/
python-packages/fastboot/setup.py
python-packages/adb/setup.py
@veloman-yunkan veloman-yunkan force-pushed the python3.13+_friendly_android-ndk-r21e branch from 8d6831f to c16951f Compare March 13, 2025 10:42
@veloman-yunkan
Copy link
Collaborator Author

The CI of this PR is broken because of the content of the PR, probably the path wrongly formatted

No, CI failed because base_deps had to be regenerated. Now it fails for a different reason (looks like python2 is used to run the build/tools/make_standalone_toolchain.py script). Let's see if enforcing python3 will finally make the CI pass.

@veloman-yunkan
Copy link
Collaborator Author

Remaining failures are unrelated to this PR:

  • Flatpak failure has been addressed by Fix Flatpak appdata.xml syntax kiwix-desktop#1351, and I don't know why it occurs here again
  • aarch64_dyn and aarch64_static both fail with the same error: meson.build:38:11: ERROR: Could not link atomics work with libatomic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

kiwix-build fail to install android ndk on recent distribution
3 participants