Skip to content

Commit 8864e56

Browse files
fenoscharislamkangmingtayiniankiwicopple
authored
docs: storage s3 protocol (supabase#22647)
* docs: storage s3 protocol * fix: edits * docs(storage): added ownership,management,custom-roles docs * chore: fix prettier * docs: fix formatting & rephrase some bits * fix: edits * add error codes documentation * update docs * Adds blog post * Adds embeds * chore: update images * parquet video * Update thumb.png * some doc updates * Adds compat list * guide: cyberduck * doc updates * ga page data * blog updates * update image * Update 2024-04-18-s3-compatible-storage.mdx * Update supabase-contributions.png * blog update * moves cyberduck to integrations * updated multipart section blog * Update 2024-04-18-s3-compatible-storage.mdx * update blog * fix link * blog updates * update blog * docs update * update announcement banners * update * update build stage * update --------- Co-authored-by: Charis <[email protected]> Co-authored-by: Kang Ming <[email protected]> Co-authored-by: Inian <[email protected]> Co-authored-by: Copple <[email protected]> Co-authored-by: Jonathan Summers-Muir <[email protected]> Co-authored-by: Francesco Sansalvadore <[email protected]> Co-authored-by: Francesco Sansalvadore <[email protected]>
1 parent c59141f commit 8864e56

File tree

30 files changed

+1567
-260
lines changed

30 files changed

+1567
-260
lines changed

apps/docs/components/Navigation/NavigationMenu/NavigationMenu.constants.ts

+25
Original file line numberDiff line numberDiff line change
@@ -1338,6 +1338,10 @@ export const storage: NavMenuConstant = {
13381338
name: 'Security',
13391339
url: undefined,
13401340
items: [
1341+
{
1342+
name: 'Ownership',
1343+
url: '/guides/storage/security/ownership',
1344+
},
13411345
{
13421346
name: 'Access Control',
13431347
url: '/guides/storage/security/access-control',
@@ -1356,6 +1360,10 @@ export const storage: NavMenuConstant = {
13561360
name: 'Resumable Uploads',
13571361
url: '/guides/storage/uploads/resumable-uploads',
13581362
},
1363+
{
1364+
name: 'S3 Uploads',
1365+
url: '/guides/storage/uploads/s3-uploads',
1366+
},
13591367
{ name: 'Limits', url: '/guides/storage/uploads/file-limits' },
13601368
],
13611369
},
@@ -1370,6 +1378,22 @@ export const storage: NavMenuConstant = {
13701378
},
13711379
],
13721380
},
1381+
{
1382+
name: 'Management',
1383+
url: undefined,
1384+
items: [
1385+
{ name: 'Copy / Move Objects', url: '/guides/storage/management/copy-move-objects' },
1386+
{ name: 'Delete Objects', url: '/guides/storage/management/delete-objects' },
1387+
],
1388+
},
1389+
{
1390+
name: 'S3',
1391+
url: undefined,
1392+
items: [
1393+
{ name: 'Authentication', url: '/guides/storage/s3/authentication' },
1394+
{ name: 'API Compatibility', url: '/guides/storage/s3/compatibility' },
1395+
],
1396+
},
13731397
{
13741398
name: 'CDN',
13751399
url: undefined,
@@ -1396,6 +1420,7 @@ export const storage: NavMenuConstant = {
13961420
name: 'Helper Functions',
13971421
url: '/guides/storage/schema/helper-functions',
13981422
},
1423+
{ name: 'Custom Roles', url: '/guides/storage/schema/custom-roles' },
13991424
],
14001425
},
14011426
{

apps/docs/content/guides/getting-started/features.mdx

+5
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,10 @@ Transform images on the fly. [Docs](/docs/guides/storage/serving/image-transform
142142

143143
Upload large files using resumable uploads. [Docs](/docs/guides/storage/uploads/resumable-uploads).
144144

145+
### S3 compatibility
146+
147+
Interact with Storage from tool which supports with the S3 protocol. [Docs](/docs/guides/storage/s3/compatibility).
148+
145149
## Edge Functions
146150

147151
### Deno Edge Functions
@@ -218,6 +222,7 @@ In addition to the Beta requirements, features in GA are covered by the [uptime
218222
| Storage | Smart CDN | `GA` | 🚧 [Cloudflare](https://www.cloudflare.com) |
219223
| Storage | Image Transformations | `GA` ||
220224
| Storage | Resumable Uploads | `GA` ||
225+
| Storage | S3 compatibility | `public alpha` ||
221226
| Edge Functions | | `beta` ||
222227
| Edge Functions | Regional Invocations | `beta` ||
223228
| Edge Functions | NPM compatibility | `beta` ||

apps/docs/content/guides/storage/debugging/error-codes.mdx

+73-6
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,79 @@
22
id: 'storage-errors-codes'
33
title: 'Error Codes'
44
description: 'Supabase Error Codes'
5+
subtitle: 'Learn about the Storage error codes and how to resolve them'
56
sidebar_label: 'Debugging'
67
---
78

9+
## Storage Error Codes
10+
11+
<Admonition type="note">
12+
We are transitioning to a new error code system. For backwards compatibility you'll still be able
13+
to see the old error codes
14+
</Admonition>
15+
16+
Error codes in Storage are returned as part of the response body. They are useful for debugging and understanding what went wrong with your request.
17+
The error codes are returned in the following format:
18+
19+
```json
20+
{
21+
"code": "error_code",
22+
"message": "error_message"
23+
}
24+
```
25+
26+
Here is the full list of error codes and their descriptions:
27+
28+
| ErrorCode | Description | StatusCode | Resolution |
29+
| ------------------------- | --------------------------------------------------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
30+
| NoSuchBucket | The specified bucket does not exist. | 404 | Verify the bucket name and ensure it exists in the system, if it exists you don't have permissions to access it. |
31+
| NoSuchKey | The specified key does not exist. | 404 | Check the key name and ensure it exists in the specified bucket, if it exists you don't have permissions to access it. |
32+
| NoSuchUpload | The specified upload does not exist. | 404 | The uploadID provided might not exists or the Upload was previously aborted |
33+
| InvalidJWT | The provided JWT (JSON Web Token) is invalid. | 401 | The JWT provided might be expired or malformed, provide a valid JWT |
34+
| InvalidRequest | The request is not properly formed. | 400 | Review the request parameters and structure, ensure they meet the API's requirements, the error message will provide more details |
35+
| TenantNotFound | The specified tenant does not exist. | 404 | The Storage service had issues while provisioning, please [Contact Support](https://supabase.com/dashboard/support/new) |
36+
| EntityTooLarge | The entity being uploaded is too large. | 413 | Verify the max-file-limit is equal or higher to the resource you are trying to upload, you can change this value on the [Project Setting](https://supabase.com/dashboard/project/_/settings/storage) |
37+
| InternalError | An internal server error occurred. | 500 | Investigate server logs to identify the cause of the internal error. If you think it's a Storage error please [Contact Support](https://supabase.com/dashboard/support/new) |
38+
| ResourceAlreadyExists | The specified resource already exists. | 409 | Use a different name or identifier for the resource to avoid conflicts. Use `x-upsert:true` header to overwrite the resource. |
39+
| InvalidBucketName | The specified bucket name is invalid. | 400 | Ensure the bucket name follows the naming conventions and does not contain invalid characters. |
40+
| InvalidKey | The specified key is invalid. | 400 | Verify the key name and ensure it follows the naming conventions. |
41+
| InvalidRange | The specified range is not valid. | 416 | Make sure that range provided is within the file size boundary and follow the [HTTP Range spec](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range) |
42+
| InvalidMimeType | The specified MIME type is not valid. | 400 | Provide a valid MIME type, ensure using the standard MIME type format |
43+
| InvalidUploadId | The specified upload ID is invalid. | 400 | The upload ID provided is invalid or missing. Make sure to provide a active uploadID |
44+
| KeyAlreadyExists | The specified key already exists. | 409 | Use a different key name to avoid conflicts with existing keys. Use `x-upsert:true` header to overwrite the resource. |
45+
| BucketAlreadyExists | The specified bucket already exists. | 409 | Choose a unique name for the bucket that does not conflict with existing buckets. |
46+
| DatabaseTimeout | Timeout occurred while accessing the database. | 504 | Investigate database performance and increase the default pool size. If this error still occurs please upgrade your instance |
47+
| InvalidSignature | The signature provided does not match the calculated signature. | 403 | Check that you are providing the correct signature format, for more information refer to [SignatureV4](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html) |
48+
| SignatureDoesNotMatch | The request signature does not match the calculated signature. | 403 | Check your credentials, access key id / access secret key / region that are all correct, refer to [S3 Authentication](/docs/guides/storage/s3/authentication). |
49+
| AccessDenied | Access to the specified resource is denied. | 403 | Check that you have the correct RLS policy to allow access to this resource |
50+
| ResourceLocked | The specified resource is locked. | 423 | This resource cannot be altered while there is a lock. Wait and try the request again |
51+
| DatabaseError | An error occurred while accessing the database. | 500 | Investigate database logs and system configuration to identify and address the database error. |
52+
| MissingContentLength | The Content-Length header is missing. | 411 | Ensure the Content-Length header is included in the request with the correct value. |
53+
| MissingParameter | A required parameter is missing in the request. | 400 | Provide all required parameters in the request to fulfill the API's requirements. The message field will contain more details |
54+
| InvalidUploadSignature | The provided upload signature is invalid. | 403 | The MultiPartUpload record was altered while the upload was ongoing, the signature do not match. Do not alter the upload record |
55+
| LockTimeout | Timeout occurred while waiting for a lock. | 423 | The lock couldn't be acquired within the specified timeout. Wait and try the request again |
56+
| S3Error | An error occurred related to Amazon S3. | - | Refer to Amazon S3 documentation or [Contact Support](https://supabase.com/dashboard/support/new) for assistance with resolving the S3 error. |
57+
| S3InvalidAccessKeyId | The provided AWS access key ID is invalid. | 403 | Verify the AWS access key ID provided and ensure it is correct and active. |
58+
| S3MaximumCredentialsLimit | The maximum number of credentials has been reached. | 400 | The maximum limit of credentials is reached. |
59+
| InvalidChecksum | The checksum of the entity does not match. | 400 | Recalculate the checksum of the entity and ensure it matches the one provided in the request. |
60+
| MissingPart | A part of the entity is missing. | 400 | Ensure all parts of the entity are included in the request before completing the operation. |
61+
| SlowDown | The request rate is too high and has been throttled. | 503 | Reduce the request rate or implement exponential backoff and retry mechanisms to handle throttling. |
62+
63+
## Legacy Error Codes
64+
65+
As we are transitioning to a new error code system, you might still see the following error format:
66+
67+
```json
68+
{
69+
"httpStatusCode": 400,
70+
"code": "error_code",
71+
"message": "error_message"
72+
}
73+
```
74+
875
Here's a list of the most common error codes and their potential resolutions:
976

10-
## 404 not_found
77+
### 404 not_found
1178

1279
Indicates that the resource is not found or you don't have the correct permission to access it
1380
**Resolution:**
@@ -16,22 +83,22 @@ Indicates that the resource is not found or you don't have the correct permissio
1683
- Ensure you include the user `Authorization` header
1784
- Verify the object exists
1885

19-
## 409 already_exists
86+
### 409 already_exists
2087

2188
Indicates that the resource already exists.
2289
**Resolution:**
2390

2491
- Use the `upsert` functionality in order to overwrite the file. Find out more [here](/docs/guides/storage/uploads/standard-uploads#overwriting-files).
2592

26-
## 403 unauthorized
93+
### 403 unauthorized
2794

2895
You don't have permission to action this request
2996
**Resolution:**
3097

3198
- Add RLS policy to grant permission. See our [Access Control docs](/docs/guides/storage/uploads/access-control) for more information.
3299
- Ensure you include the user `Authorization` header
33100

34-
## 429 too many requests
101+
### 429 too many requests
35102

36103
This problem typically arises when a large number of clients are concurrently interacting with the Storage service, and the pooler has reached its `max_clients` limit.
37104

@@ -40,7 +107,7 @@ This problem typically arises when a large number of clients are concurrently in
40107
- Increase the max_clients limits of the pooler.
41108
- Upgrade to a bigger project compute instance [here](https://supabase.com/dashboard/project/_/settings/addons).
42109

43-
## 544 database_timeout
110+
### 544 database_timeout
44111

45112
This problem arises when a high number of clients are concurrently using the Storage service, and Postgres doesn't have enough available connections to efficiently handle requests to Storage.
46113

@@ -49,7 +116,7 @@ This problem arises when a high number of clients are concurrently using the Sto
49116
- Increase the pool_size limits of the pooler.
50117
- Upgrade to a bigger project compute instance [here](https://supabase.com/dashboard/project/_/settings/addons).
51118

52-
## 500 internal_server_error
119+
### 500 internal_server_error
53120

54121
This issue occurs where there is a unhandled error.
55122
**Resolution:**
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
id: 'storage-management'
3+
title: 'Copy Objects'
4+
description: 'Learn how to copy and move objects'
5+
subtitle: 'Learn how to copy and move objects'
6+
sidebar_label: 'Copy / Move Objects'
7+
---
8+
9+
## Copy objects
10+
11+
You can copy objects between buckets or within the same bucket. Currently only objects up to 5 GB can be copied using the API.
12+
13+
When making a copy of an object, the owner of the new object will be the user who initiated the copy operation.
14+
15+
### Copying objects within the same bucket
16+
17+
To copy an object within the same bucket, use the `copy` method.
18+
19+
```javascript
20+
await supabase.storage.from('avatars').copy('public/avatar1.png', 'private/avatar2.png')
21+
```
22+
23+
### Copying objects across buckets
24+
25+
To copy an object across buckets, use the `copy` method and specify the destination bucket.
26+
27+
```javascript
28+
await supabase.storage.from('avatars').copy('public/avatar1.png', 'private/avatar2.png', {
29+
destinationBucket: 'avatars2',
30+
})
31+
```
32+
33+
## Move objects
34+
35+
You can move objects between buckets or within the same bucket. Currently only objects up to 5GB can be moved using the API.
36+
37+
When moving an object, the owner of the new object will be the user who initiated the move operation. Once the object is moved, the original object will no longer exist.
38+
39+
### Moving objects within the same bucket
40+
41+
To move an object within the same bucket, you can use the `move` method.
42+
43+
```javascript
44+
const { data, error } = await supabase.storage
45+
.from('avatars')
46+
.move('public/avatar1.png', 'private/avatar2.png')
47+
```
48+
49+
### Moving objects across buckets
50+
51+
To move an object across buckets, use the `move` method and specify the destination bucket.
52+
53+
```javascript
54+
await supabase.storage.from('avatars').move('public/avatar1.png', 'private/avatar2.png', {
55+
destinationBucket: 'avatars2',
56+
})
57+
```
58+
59+
## Permissions
60+
61+
For a user to move and copy objects, they need `select` permission on the source object and `insert` permission on the destination object. For example:
62+
63+
```sql
64+
create policy "User can select their own objects (in any buckets)"
65+
on storage.objects
66+
for select
67+
to authenticated
68+
using (
69+
owner_id = (select auth.uid())
70+
);
71+
72+
create policy "User can upload in their own folders (in any buckets)"
73+
on storage.objects
74+
for insert
75+
to authenticated
76+
with check (
77+
(storage.folder(name))[1] = (select auth.uid())
78+
);
79+
```
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
id: 'storage-management'
3+
title: 'Delete Objects'
4+
description: 'Learn about deleting objects'
5+
subtitle: 'Learn about deleting objects'
6+
sidebar_label: 'Delete Objects'
7+
---
8+
9+
When you delete one or more objects from a bucket, the files are permanently removed and not recoverable. You can delete a single object or multiple objects at once.
10+
11+
<Admonition type="note">
12+
13+
Deleting objects should always be done via the **Storage API** and NOT via a **SQL query**. Deleting objects via a SQL query will not remove the object from the bucket and will result in the object being orphaned.
14+
15+
</Admonition>
16+
17+
## Delete objects
18+
19+
To delete one or more objects, use the `remove` method.
20+
21+
```javascript
22+
await supabase.storage.from('bucket').remove(['object-key-1', 'object-key-2'])
23+
```
24+
25+
## RLS
26+
27+
To delete an object, the user must have the `delete` permission on the object. For example:
28+
29+
```sql
30+
create policy "User can delete their own objects"
31+
on storage.objects
32+
for delete
33+
TO authenticated
34+
USING (
35+
owner_id = (select auth.uid())
36+
);
37+
```

0 commit comments

Comments
 (0)