Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UploadFilter should evaluate directories, not just files #92

Open
graebm opened this issue Jan 21, 2025 · 0 comments
Open

UploadFilter should evaluate directories, not just files #92

graebm opened this issue Jan 21, 2025 · 0 comments

Comments

@graebm
Copy link
Contributor

graebm commented Jan 21, 2025

Currently, the UploadFilter only evaluates files, and the default filter does NOT exclude hidden directories (source).

If a user wanted to exclude files in hidden directories, their filter would have to crawl up the filepath for ANY segment starting with ".". This filter would be run on every single file beneath the directory they didn't want to upload.

A more intuitive experience would be to run the UploadFilter on each directory before recursing into it (user just needs to check if name starts with "."). And if false is returned, don't recurse into the directory at all. This is also far more efficient.

This change will likely require transfer manager to write its own directory walking code (currently uses walkdir), which isn't hard, but just bringing it up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant