Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct minimum versions in dependency version ranges #1272

Merged
merged 16 commits into from
Mar 11, 2025

Conversation

dlqqq
Copy link
Member

@dlqqq dlqqq commented Mar 10, 2025

Description

Details

  • Removes 2 unused dependencies: typing_extensions and aiosqlite.
  • Bumps traitlets>=5.0 to traitlets>=5.6 to match JupyterLab 4's dependencies.
  • Bumps jupyter_server>=1.6 to jupyter_server>=2.11.1 to match jupyter_server_ydoc>=3 dependencies.
    • When backporting, we should try to lower this to jupyter_server>=2.4 to match JupyterLab 4's dependencies.
  • Bumps click>=8.0.0 to click>=8.1.0. click==8.0.0 was released almost 4 years ago and is a very common dependency. Bumping this floor speeds up minimum versions tests in CI.
  • Bumps pydantic>=2.0.0 to pydantic>=2.10.0.
  • Drops Python 3.8 support, since it has reached end-of-life and we are not testing that in our CI workflows.
  • Removes importlib_metadata package, as the importlib.metadata module is now part of the standard library in Python 3.8+.
    • Not possible because entry_points().select(...) fails in Python 3.9, since the API wasn't finalized until Python 3.10.

@dlqqq dlqqq added the maintenance Change related to maintenance of the repository label Mar 10, 2025
@dlqqq dlqqq self-assigned this Mar 10, 2025
@dlqqq
Copy link
Member Author

dlqqq commented Mar 10, 2025

The latest CI failure is being caused by typing_extensions. The typing_extensions dependency was added in #192, but does not appear to be used. I'll remove this and see what happens.

@dlqqq dlqqq changed the title Fix minimum dependencies CI workflow Raise dependency version floors to be self-consistent Mar 10, 2025
@dlqqq dlqqq changed the title Raise dependency version floors to be self-consistent Raise dependency version floors to be usable Mar 10, 2025
@dlqqq dlqqq added the bug Bugs reported by users label Mar 10, 2025
@dlqqq
Copy link
Member Author

dlqqq commented Mar 10, 2025

One issue is that the minimum dependencies job is now taking 20+ minutes. I think the runner may be struggling with memory use when trying to resolve the dependency set, since network downloads of PyPI packages shouldn't take this long.

@dlqqq
Copy link
Member Author

dlqqq commented Mar 10, 2025

I found a couple more issues with our dependencies that may be causing this job to take forever:

  1. aiosqlite is required but never used.
  2. ollama>=0.4.0 requires pydantic>=2.9.0: https://github.com/ollama/ollama-python/blob/v0.4.0/pyproject.toml

@dlqqq
Copy link
Member Author

dlqqq commented Mar 10, 2025

The qianfan package has a huge list of dependencies which may also be slowing down the resolver: https://github.com/baidubce/bce-qianfan-sdk/blob/main/python/pyproject.toml

@dlqqq dlqqq added the scope:dependencies Issues concerning dependencies label Mar 10, 2025
@dlqqq
Copy link
Member Author

dlqqq commented Mar 10, 2025

@dlqqq dlqqq force-pushed the fix-min-deps-workflow branch from f57f1f8 to 13a43dd Compare March 10, 2025 22:52
@dlqqq
Copy link
Member Author

dlqqq commented Mar 10, 2025

Looks like bumping pydantic and dropping importlib_metadata really helped speed up the dev-install step for jupyter_ai_magics. The jupyter_ai package is still taking a long time to dev-install however. Will check in this PR tomorrow, have to head off today.

@dlqqq
Copy link
Member Author

dlqqq commented Mar 11, 2025

Bumping the pydantic, click, and jupyter_server version floors allowed pip to quickly resolve the dependency set and complete installation in the CI.

Unfortunately, even though importlib.metadata provides entry points in Python 3.8+, the API wasn't finalized until Python 3.10.

Screenshot 2025-03-11 at 10 04 11 AM

CI is able to install the dependencies, but the tests fail because the entry_points().select(...) method only exists in Python 3.10+, not Python 3.9. 🤦

I'll likely have to add importlib_metadata back to fix Python 3.9 support. We can finally remove this dependency once Python 3.9 reaches EOL in October of this year.

@dlqqq dlqqq force-pushed the fix-min-deps-workflow branch from 2d5d6f5 to 3ff82b2 Compare March 11, 2025 17:36
@dlqqq dlqqq force-pushed the fix-min-deps-workflow branch from 3ff82b2 to 8f4cd5d Compare March 11, 2025 17:36
Copy link
Collaborator

@srdas srdas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All changes look good and the CI is also passing with these changes. Thanks for sorting out these dependency issues!

@dlqqq
Copy link
Member Author

dlqqq commented Mar 11, 2025

@krassowski Thank you for contributing to maintainer-tools to add this to our CI. I never knew that an excessively loose version range can trigger this error for users in older environments:

"RuntimeError: maximum recursion depth exceeded while calling a Python object"

Thanks to your contributions, our minimum dependency versions will now be correct & verifiable (as of v2.30.0+). pip install jupyter_ai should be faster & more reliable for users in older environments now. This is awesome! 🤗

@dlqqq dlqqq changed the title Raise dependency version floors to be usable Correct minimum versions in dependency version ranges Mar 11, 2025
@dlqqq dlqqq merged commit 110311b into jupyterlab:main Mar 11, 2025
10 checks passed
@dlqqq
Copy link
Member Author

dlqqq commented Mar 11, 2025

@meeseeksdev please backport to 2.x

meeseeksmachine pushed a commit to meeseeksmachine/jupyter-ai that referenced this pull request Mar 11, 2025
dlqqq added a commit that referenced this pull request Mar 11, 2025
…cy version ranges) (#1275)

* Backport PR #1272: Correct minimum versions in dependency version ranges

* loosen jupyter_server version range for 2.x

---------

Co-authored-by: David L. Qiu <[email protected]>
@krassowski
Copy link
Member

Thank you for the work here, I appreciate it was a deep hole! The most important thing is it will help to catch some issues when using a function from a new version of a package and forgetting to bump the dependency constraint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bugs reported by users maintenance Change related to maintenance of the repository scope:dependencies Issues concerning dependencies
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Minimum dependencies CI job is failing
3 participants