Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reading data from file requires s3:ListAllMyBuckets permission #890

Open
kxm-kstlr opened this issue Mar 18, 2025 · 3 comments · May be fixed by #891
Open

Reading data from file requires s3:ListAllMyBuckets permission #890

kxm-kstlr opened this issue Mar 18, 2025 · 3 comments · May be fixed by #891
Labels
bug Something isn't working needs triage The issue needs to be triaged, before work can commence

Comments

@kxm-kstlr
Copy link

Bug Description

I cannot read the contents of an object, even though I have the s3:GetObject, as well as the s3:ListBucket permission for it. Additionally the s3:ListAllMyBuckets permission is needed, otherwise the access is forbidden.

Expected Behavior

A user who is allowed to access an object should be able to access it without additional permissions. In our scenario it is not feasible to give the
ListAllMyBuckets permission.

Environment

  • s3fs-nio version: 2.0.0
  • OS: Windows 10
  • JDK: 8

Additional context

This behavior is caused in S3SeekableByteChannel:85 when calling getBucket. This method tries to get all buckets and then filter for the one needed, thus needing the s3:ListAllMyBuckets permission.

Proposed Solution

Remove the call of getBucket and instead retrieve the name of the bucket directly from the path object by calling getBucketName on it.
Also, I want to open a pull request with the proposed solution shortly.
@kxm-kstlr kxm-kstlr added bug Something isn't working needs triage The issue needs to be triaged, before work can commence labels Mar 18, 2025
@steve-todorov
Copy link
Collaborator

Hey,

Thanks for reporting this.

The s3:listAllMyBuckets permission is needed to comply with the Java NIO interfaces. With Java NIO you can "list all drives" -- a Windows analogy would be listing C:, D:. Using the Ss3fs-nio proider you can list all buckets.

I don't think we're using the listAllMyBuckets REST API endpoint anywhere else but to list all buckets. However I need to double check the code to be sure. If we are -- then we might need to look into ways to avoid it if possible.

@kxm-kstlr kxm-kstlr linked a pull request Mar 18, 2025 that will close this issue
Open
7 tasks
@kxm-kstlr
Copy link
Author

Hey,
thanks for the quick reply.

The issue is not related to behavior of listAllMyBuckets, but its use for retrieving data in S3SeekableByteChannel.
My change in #891 makes use of the bucket name in S3Path. This solves my issue of having to give a user the permission s3:ListAllMyBuckets to access the objects of that bucket.

@kxm-kstlr
Copy link
Author

I just added a commit to my PR replacing the getBucket call in S3FileSystemProvider with a new method hasBucket. This will also avoid the need for the S3:ListAllMyBuckets permission in this call. Also I think it is more logical to call hasBucket to check if a bucket exists instead of calling getBucket to get all buckets and then filter for one specific bucket to find out if it exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working needs triage The issue needs to be triaged, before work can commence
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Get bucket name directly from path to avoid calling 'listBuckets' kxm-kstlr/s3fs-nio
2 participants
@steve-todorov @kxm-kstlr