[MAINTENANCE] Multiple versions of crates #1948
Comments
eliaspekkala
added
status: needs triage
type: maintenance
distransient
added
status: in progress
team: engine
and removed
status: needs triage
labels
|
I have now measured the old crates percentages of the total build time using the data from: Here's the results:
My build time is usually around 120s, so if these old crate versions are removed |
|
Thanks! Shall slowly chew through the ecosystem ( cc: @Veykril if you ever feel bored |
|
Related: amethyst/rendy#203 |
Current state: 45536af (2020-02-15)
Somehow it's gotten worse |
|
This issue has been automatically marked as stale because it has not had recent activity. Maybe it's time to revisit this? |
|
Yes, thank you stalebot! I added a reference to this issue in #2407 where we are tracking ways we can improve CI -- I think we should clean this up once as much as we can and then add some CI to watch for regressions. |
|
#2590 has gotten build time to around 100s on ryzen 1950x from clean. rendy and atelier-assets trees have lots of duplication but because they are internal crates we can manage their dependency updates with github actions / dependabot. already have dependabot running for rendy. I've also disabled some unused default-features on crates to shorten build times a bit. I will be adding dependabot config from rendy to amethyst repo. |
|
Currently, we have heavy issues with bloat and having too many dependencies. This will be addressed, but will take a lot of time and will never be "complete". As such, instead of tracking progress in this issue, we'll progressively work to reduce our dependency list and update it whenever we can. To encourage this, we will encourage the use of smaller dependencies whenever possible. We will also notify people with merge access that pull requests that add dependencies should be carefully looked at starting from now. We will not be tracking tasks here (because the list would be endless and keep changing), but that doesn't mean you can't help! |
Description
In my investigation to reduce Amethyst build times I have found that there are many crates in Amethyst's dependency tree that Cargo isn't able to resolve into a single version. This means that Cargo will build and include several versions of some crates.
I have created a command to automatically find and list crates with many versions. The command is
cargo tree --duplicate --prefix-depth | grep '^[\0]' | sort | cut -c3-.To run the command you need
cargo tree. You also need to specify the features you want to include in Amethyst's Cargo.toml.Note that:
cargo updatebefore the commandThe command output:
This shows that are 22 crates that have multiple versions included in Amethyst.
It also shows that there are 26 older versions of crates that could potentially be removed with some upgrades/updates in Amethyst or in the crates in its dependency tree.
Steps to remove multiple versions
The table below shows the old versions that we want to remove and the steps to remove them.
You can use
cargo treeorcargo tree --duplicateto investigate the steps in the cases where I haven't figured them out yet.Note: Some of the steps will require the crate author to do the updates and then to create a release on crates.io.
cgmathwithnalgebraingenmesh.nalgebraingenmeshthen bump genmesh (needs release first) in amethyst_rendyProgess
This table shows the old crate versions that have been removed, the steps taken, and the PR where it was done.
The PR #1952 also reduced the number of steps necessary to remove some of the crates.
Reason
To reduce build times.
Impact
I have not measured the benefits in terms of build times.
Additional Information
Use
cargo treeto see Amethyst's dependency tree and the origin of the crates.The text was updated successfully, but these errors were encountered: