Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(biome_fs): make paths absolute to correctly match against globs #5017

Merged
merged 2 commits into from
Feb 13, 2025

Conversation

Conaclos
Copy link
Member

@Conaclos Conaclos commented Feb 2, 2025

Summary

We recently added a normalization step before matching against glob pattern.
The normalization step strips the working directory from checked evaluated path.
Some passed paths are not absolute making it impossible to properly normalize.

This PR fixes the issue, by converting relative paths passed to the CLI into absolute paths.

I first modified the traverse_inputs function in biome_cli/src/traverse.rs.
The fix was correct, however this makes fails the implementation of MemoryFs that assumes relative paths.
Thus, I moved the fix to OsTraversalScope::evaluate in biome_fs/src/fs/os.rs.
This may make the fix less performant.

It is not the first time I encounter issues with our memory FS.
I wonder if we should not remove this abstraction and directly use a temporary folder for each test.
This could have the nice effect of actually testing real code paths and to identify bugs like the one this PR fixes.

Test Plan

I manually tested the fix.
It isn't possible to test it with memory fs because the change is in the os filesystem.

@github-actions github-actions bot added A-CLI Area: CLI A-Core Area: core labels Feb 2, 2025
@Conaclos Conaclos requested review from a team February 2, 2025 18:35
Copy link

codspeed-hq bot commented Feb 2, 2025

CodSpeed Performance Report

Merging #5017 will not alter performance

Comparing conaclos/absolutize-paths (bd79c92) with main (9280cba)

Summary

✅ 94 untouched benchmarks

Copy link
Contributor

@arendjr arendjr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do wonder if we can’t fix MemoryFS instead. I recently wrote a few tests with it where I used absolute paths instead and it worked fine. So I suspect the issue isn’t so much that absolute paths can’t work there, just that many tests are written to assume relative paths. I understand it would be painful to update those tests, but it would give us better coverage and better performance.

@@ -234,6 +235,12 @@ impl<'scope> OsTraversalScope<'scope> {

impl<'scope> TraversalScope<'scope> for OsTraversalScope<'scope> {
fn evaluate(&self, ctx: &'scope dyn TraversalContext, path: Utf8PathBuf) {
let path = match std::path::Path::new(&path).absolutize() {
Ok(std::borrow::Cow::Owned(absolutized)) => {
Utf8PathBuf::from_path_buf(absolutized).unwrap_or(path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using unwrap_or() may make it very hard to debug if there is ever an issue with an absolute path, since it means we’ll unexpectedly pass a relative path instead. I’d just use .expect() here.

Copy link
Member

@ematipico ematipico left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would appreciate we could, at least, add one test that covers the issue we're trying to fix.

We already have tests that use the OS file system:

let _ = remove_dir_all(&root_path);
create_dir_all(&subdir1_path).unwrap();
create_dir_all(&subdir2_path).unwrap();
#[cfg(target_family = "unix")]
{
symlink(&subdir2_path, subdir1_path.join("symlink1")).unwrap();
symlink(subdir1_path, subdir2_path.join("symlink2")).unwrap();
}
#[cfg(target_os = "windows")]
{
check_windows_symlink!(symlink_dir(&subdir2_path, &subdir1_path.join("symlink1")));
check_windows_symlink!(symlink_dir(subdir1_path, subdir2_path.join("symlink2")));
}
let result = run_cli(
DynRef::Owned(Box::new(OsFileSystem::new(root_path.clone()))),
&mut console,
Args::from([("lint"), (root_path.display().to_string().as_str())].as_slice()),
);

@Conaclos
Copy link
Member Author

Conaclos commented Feb 5, 2025

@ematipico
Thanks for the pointer.

However, it is not possible to declare a working directory with run_cli_with_dyn_fs.
I don't know if it is easily doable.
Any suggestion ?

@ematipico
Copy link
Member

The argument that you pass to OsFileSystem::new is the working directory

let result = run_cli_with_dyn_fs(
Box::new(OsFileSystem::new(root_path.clone())),
&mut console,
Args::from(["lint", root_path.as_ref()].as_slice()),

pub fn new(working_directory: PathBuf) -> Self {
Self {
working_directory: Some(working_directory),
configuration_resolver: AssertUnwindSafe(Resolver::new(ResolveOptions {
condition_names: vec!["node".to_string(), "import".to_string()],
extensions: vec![".json".to_string(), ".jsonc".to_string()],
..ResolveOptions::default()
})),
}
}

Base automatically changed from next to main February 12, 2025 11:41
@Conaclos Conaclos force-pushed the conaclos/absolutize-paths branch 3 times, most recently from b603e76 to f228d10 Compare February 13, 2025 17:27
dbg!(&path);
let path = match std::path::Path::new(&path).absolutize() {
Ok(std::borrow::Cow::Owned(absolutized)) => Utf8PathBuf::from_path_buf(absolutized)
.expect("Absolute path must be correctly parsed"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully this panic won't bite us

@Conaclos Conaclos force-pushed the conaclos/absolutize-paths branch 3 times, most recently from 64552d3 to bd79c92 Compare February 13, 2025 17:30
@Conaclos
Copy link
Member Author

I do wonder if we can’t fix MemoryFS instead. I recently wrote a few tests with it where I used absolute paths instead and it worked fine. So I suspect the issue isn’t so much that absolute paths can’t work there, just that many tests are written to assume relative paths. I understand it would be painful to update those tests, but it would give us better coverage and better performance.

I agree. Feel free to investigate the case once the PR merged.
I added a FIXME in the code.

@Conaclos
Copy link
Member Author

Conaclos commented Feb 13, 2025

I added an end-to-end test.

Once the CI passes, I will merge the PR.

@Conaclos Conaclos merged commit 31d5416 into main Feb 13, 2025
11 of 12 checks passed
@Conaclos Conaclos deleted the conaclos/absolutize-paths branch February 13, 2025 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-CLI Area: CLI A-Core Area: core
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants