Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dependabot support #2512

Open
mjclarke94 opened this issue Mar 18, 2024 · 23 comments
Open

Add dependabot support #2512

mjclarke94 opened this issue Mar 18, 2024 · 23 comments
Labels
integration Integration into another tool wish Not on the immediate roadmap

Comments

@mjclarke94
Copy link

I'm not quite sure on whether this is a rye or a uv issue, but it would be broadly useful to have dependabot understand rye/uv backed projects as a target for dependabot updates. The current known targets are pip and poetry (which you leave as pip, they just have support for a poetry-style pyproject file).

Whether there is some way of tricking dependabot using the pip compatibility layer, or if it's better to have bespoke handlers I'm not sure!

@zanieb
Copy link
Member

zanieb commented Mar 18, 2024

We don't have a lockfile standard here at uv so I think that the pip dependabot backend should just work? I'm not sure there's more for us to do at this time. This seems more relevant for rye.

@mjclarke94
Copy link
Author

Fair enough! As a secondary point, it might be useful to work with the dependabot maintainers to allow them to use uv for dependency resolution. We have some repositories which dependabot times out trying to solve, and uv solves pretty much instantaneously.

Given they explicitly support poetry (including generating a lock file), there's definitely precedent for it, and the goal of pip compatibility means that this could just be a straight swap in some repositories.

@zanieb zanieb added wish Not on the immediate roadmap integration Integration into another tool labels Mar 21, 2024
@notatallshaw
Copy link
Collaborator

We have some repositories which dependabot times out trying to solve, and uv solves pretty much instantaneously.

Do you have a concrete example?

Not uv related, but I have a significant improvement to Pip's resolution for complex backtracking cases (https://github.com/notatallshaw/pip/tree/prefer-conflicting-causes), and it would be good to know as many real world cases it solves or not.

@avilaton
Copy link

I tried spending a little time on this today and got some feedback from dependabot's CI here https://github.com/dependabot/dependabot-core/actions/runs/9581241755/job/26417616046

One error appears to be easy to fix, there is a check to control that there is an error trying to install a dependency that should fail.

The other I have to look into but here is the message we get from uv when dependabot tries to use it as a direct replacement for pip-compile

"error: invalid value 'pyyaml==6.0.1' for '--upgrade-package <UPGRADE_PACKAGE>': Not a valid package or extra name: \"pyyaml==6.0.1\". Names must start and end with a letter or digit and may only contain -, _, ., and alphanumeric characters.\n\nFor more information, try '--help'."

@avilaton
Copy link

avilaton commented Jul 3, 2024

Made a bit more progress and found that there is an existing issue here to fix what I encountered above #1964, writing it down for future generations.

@avilaton
Copy link

The above issue is solved, one step closer! We are now blocked with a tiny header difference issue #5031 which is already merged and will be out with next release, thanks @skshetry .

I've mentioned it in another issue but I'm holding from the autogenerated by uv string in the header to swap pip-compile for uv pip compile for the lack of any better idea and since the header still receives updates, I'd like for this to be taken into account.

@skshetry
Copy link
Contributor

Another alternative could to be parse the compile_command. That is what renovatebot does.

kdeldycke added a commit to kdeldycke/workflows that referenced this issue Jul 16, 2024
Dependabot does not support uv yet: astral-sh/uv#2512
@albertferras-vrf
Copy link

Is there any update on this or an estimate on when uv will support dependabot? I can see the issues you mentioned @avilaton are resolved.

@avilaton
Copy link

Is there any update on this or an estimate on when uv will support dependabot? I can see the issues you mentioned @avilaton are resolved.

I think the uv side of it is done, but need help adding more test coverage on the dependabot PR I submitted and haven't had time to do it. If you have time please have a look at it, all help is welcome!

@danieleades
Copy link
Contributor

Maybe I'm reading it wrong but this issue seems to have evolved a little after it was originally created.
I'm blocked from adopting UV until dependably supports bumping dev dependencies. Is this supported? Is support coming?

@zanieb
Copy link
Member

zanieb commented Aug 21, 2024

See #6236 — basically it's not supported yet. I think they're working on it still. Renovate supports this.

@jonjanego
Copy link

👋 from the GitHub Dependabot team. There's a PR in dependabot-core that aims to help support uv. @zanieb in case you didn't notice the tag there, i'm pinging you here to see if you could take a look :)

Thanks!

@sstrullmyer
Copy link

Noting this just shipped GA: https://github.blog/changelog/2025-03-13-dependabot-version-updates-now-support-uv-in-general-availability/

@markhallen
Copy link

Hello from the GitHub Dependabot team 👋

I have made some updates to how we handle uv.lock files and merged today.

We have a generated bump PR for the httpx dependency on our testing repo.

I generated the original uv.lock file on MacOS. The subsequent run of uv lock --upgrade-package httpx in the Dependabot action also removed some wheels for the unrelated dependency charset-normalizer.

Is this expected behaviour? Any feedback on this would be gratefully received. Thank you!

@zanieb
Copy link
Member

zanieb commented Mar 19, 2025

Thank you for reaching out!

I took a look at that pull request (in Dependabot) and it looks better.

What uv version was the lockfile generated with originally? We may have added additional filtering of wheels that cannot be selected, e.g., as in #12299

@zanieb
Copy link
Member

zanieb commented Mar 19, 2025

On a separate note, I see you're testing against a dependency that is also pinned in the pyproject.toml

--- a/pyproject.toml
+++ b/pyproject.toml
@@ -5,6 +5,6 @@ description = "Add your description here"
 readme = "README.md"
 requires-python = ">=3.9"
 dependencies = [
-    "httpx==0.20",
+    "httpx==0.28.1",
     "requests==2.31.0",
 ]

I don't think this is a regular pattern, since that's the point of the uv.lock file. It'd be good to test with a constrained version too, e.g., httpx>=0.20.

@zanieb
Copy link
Member

zanieb commented Mar 19, 2025

It looks like somehow the requires-python range is being changed from >=3.9 to >=3.13 and we're dropping the old wheels. Are you changing the Python version range? If so, why?

@markhallen
Copy link

It looks like somehow the requires-python range is being changed from >=3.9 to >=3.13 and we're dropping the old wheels. Are you changing the Python version range? If so, why?

We are not intentionally changing it and actually have code to maintain it in case it is inadvertently updated.

I'm going to look into what is happening here. Thanks

@zanieb
Copy link
Member

zanieb commented Mar 19, 2025

It seems like this occurs at https://github.com/dependabot/dependabot-core/blob/59943c098d9a28736a2f3ac2945faf3048e662ed/uv/lib/dependabot/uv/file_updater/lock_file_updater.rb#L166-L170

You don't need to do that. The project metadata needs to be preserved for a correct lock.

@konstin
Copy link
Member

konstin commented Mar 19, 2025

For general semantics, uv lock will look at the pyproject.toml and at uv.lock and only make those change required to align the lockfile with pyproject.toml. This means if a ==-pinned dependency is updated in pyproject.toml, uv lock will make the relevant minimal change to uv.lock. uv lock does no change if it doesn't have to, so e.g. a plain httpx requirement won't be updated. uv lock --upgrade-package httpx otoh does update the lockfile entry for a plain httpx requirement to the latest version compatible with the rest of the pyproject.toml.

For the Python version, we use project.requires-python to determine which versions are allowed (their Requires-Python must start at least as low as the main project's one). Separately, we need a Python interpreter during locking for the case that we need to build a package. The default logic for discovering the Python interpreter is multi-layered (different sources for interpreters across platforms), but you can pass a specific one --python.

@markhallen
Copy link

It seems like this occurs at https://github.com/dependabot/dependabot-core/blob/59943c098d9a28736a2f3ac2945faf3048e662ed/uv/lib/dependabot/uv/file_updater/lock_file_updater.rb#L166-L170

You don't need to do that. The project metadata needs to be preserved for a correct lock.

I've raised a PR to remove this.

@markhallen
Copy link

markhallen commented Mar 20, 2025

Thanks for the comment @konstin

Separately, we need a Python interpreter during locking for the case that we need to build a package. The default logic for discovering the Python interpreter is multi-layered (different sources for interpreters across platforms), but you can pass a specific one --python.

Can we do something similar to uv lock --upgrade-package ... that does not try to build or download the packages and only update the lock file or even just returning the updated lock file contents? This is an approach used for other package managers in Dependabot.

@konstin
Copy link
Member

konstin commented Mar 20, 2025

Technically, you can use --no-build to not build any packages. But this means that you won't be able to resolve any resolutions with packages that only have source distribution (with Metadata Version <2.2, which is still common), so we highly recommend running the regular uv lock --upgrade-package. uv tries hard to just update the lockfile (it's what makes uv fast), but the historic lack of reliable metadata forces us to sometimes build just to extract dependency information.

Usually, we also don't download the entire wheels, given PEP 658 or Range Request support for metadata, but for the (rare) indexes that don't support either, we have no option but to download an entire wheel and read the METADATA from it.

Both of those caveats should be generally the same for pip/pip-tools and poetry (pip for example has --only-binary :all:). We expose those options more prominently, we're the only tool not written in Python, so for us, running Python code is an explicit extra step and not just a subprocess.check_call([sys.executable, ...]).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integration Integration into another tool wish Not on the immediate roadmap
Projects
None yet
Development

No branches or pull requests