Skip to content

Fix slow incremental builds (especially on Windows) #17511

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Apr 3, 2025
Merged

Conversation

philipp-spiess
Copy link
Member

@philipp-spiess philipp-spiess commented Apr 2, 2025

Fixes: #16911
Fixes: #17522

When debugging this issue, we noticed that the resolve_glob() function from detect_sources.rs was really slow, specifically on Windows. It turns out that we're now computing the list of globs that we use to attach the file system watchers repeatedly on every change instead of doing it only once per compiler setup. This changed with the updates we added to support @source not "…"; in 4.1.

More specifically, we noticed that the way we compute these sources is causing a recursive scan through all folders inside your working directory (including node_modules/ and .git/). This happens because that logic uses a vanilla file-system walker and not our ignore crate for this.

When we started investigating it, our first reaction was to change both Vite and PostCSS to use the new normalized_sources list instead. This is a list of globs to watch that is computed solely on the input sources and does not require any file-system access and it's what we use in the CLI as well. Think of it as being a **/* rule for the current working-directory in the basic auto-content case.

When we made this change, to our surprise, nothing in our existing integration test broke. From memory, though, we remembered that it's important for PostCSS though to not register a blanket watcher for the current working-directory. It took us quite a bit of tinkering until we were able to craft an example that would cause an endless re-render loop with this patch applied. We've now added a new integration test to have this context in the repository for the future but also it meant that we have to go back to creating a "smarter" list of globs for the file system watcher again.

To fix the regressions, this PR now does two things:

  1. We now cache the list of calculated globs again since it's based only on input form the CSS file. This means that rebuilds are instant as we don't have to compute these globs anymore. This was the same behavior that we had in previous 4.0 releases.
  2. Furthermore we also updated the file system walk to include our new globally ignored content directories. By bailing on any subdirectories in node_modules and .git, that walk is now much faster. We also made sure that you can still add a custom source rule to point inside a node_modules folder and it will still pick up changes like you would suggest (e.g. via @source "../node_modules/my-ui-lib";).

The results? This PR fixes slow rebuilds on Windows where rebuilds go from ~2s to ~20ms.

We suspect that this issue is more pronounced on Windows because of a slower file system in general.

Test plan

Tested it on a reproduction with the following results:

Before:

Screen.Recording.2025-04-03.at.16.35.47.mov

After:

Screen.Recording.2025-04-03.at.16.41.25.mov

Zooming in on the times, it looks like this:

Before After
image image

But with these changes:

We also tested this on Windows with the following results:

Before After
image image

[ci-all]

@RobinMalfait RobinMalfait changed the title Fix/slow scans Fix slow builds (especially on Windows) Apr 2, 2025
@RobinMalfait RobinMalfait changed the title Fix slow builds (especially on Windows) Fix slow incremental builds (especially on Windows) Apr 3, 2025
@RobinMalfait RobinMalfait force-pushed the fix/slow-scans branch 8 times, most recently from ace59e9 to 60b6d8b Compare April 3, 2025 14:32
@RobinMalfait RobinMalfait marked this pull request as ready for review April 3, 2025 14:53
@RobinMalfait RobinMalfait requested a review from a team as a code owner April 3, 2025 14:53
@RobinMalfait RobinMalfait enabled auto-merge (squash) April 3, 2025 15:06
@RobinMalfait RobinMalfait disabled auto-merge April 3, 2025 15:13
@RobinMalfait RobinMalfait merged commit 4200a1e into main Apr 3, 2025
18 of 19 checks passed
@RobinMalfait RobinMalfait deleted the fix/slow-scans branch April 3, 2025 15:13
@FragsterAt
Copy link

What about the case when a new file appears after starting dev server? (for example new vue sfc component)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants