Fixed invalid parameter exception for the table aws_s3_object Closes #2381

[ParthaI]

Integration test logs

Logs

N/A

Example query results

Results

Before Fix:

Error: aws: operation error S3: GetObjectAttributes, https response error StatusCode: 400, RequestID: QHYFJR9C3NSGZD6K, HostID: NQCelh3oQxOyMkUFyUATsdxY3VhIggAHSKGhKAv6PiVdBt1lLniV00YLzyQEknaR31gGSI6bmrROSUtqEk68W7xh20t3Onm+, api error InvalidRequest: The object was stored using a form of Server Side Encryption. The correct parameters must be provided to retrieve the object. (SQLSTATE HV000)

+-----+-----+-------------+---------------+---------------+------------+---------------+------+--------------------+---------------+----------------+-----------------+---------------+-----------------+---------------------+------------>
| key | arn | bucket_name | last_modified | storage_class | version_id | accept_ranges | body | bucket_key_enabled | cache_control | checksum_crc32 | checksum_crc32c | checksum_sha1 | checksum_sha256 | content_disposition | content_enc>
+-----+-----+-------------+---------------+---------------+------------+---------------+------+--------------------+---------------+----------------+-----------------+---------------+-----------------+---------------------+------------>
+-----+-----+-------------+---------------+---------------+------------+---------------+------+--------------------+---------------+----------------+-----------------+---------------+-----------------+---------------------+------------>

Time: 3.1s. Rows returned: 0.

Error: aws: operation error S3: HeadObject, https response error StatusCode: 400, RequestID: CSV1Q9SH7AP82XGT, HostID: 4W40JHPNA3AFMUAUsnPREtY3cpxEyZ/fbrxgQojaOSTpHGjZlMI/ldNxs3ha0FiVpE5Rt6C3Cv8=, api error BadRequest: Bad Request (SQLSTATE HV000)

+-----+-----+-------------+---------------+---------------+------------+---------------+------+--------------------+---------------+----------------+-----------------+---------------+-----------------+---------------------+------------>
| key | arn | bucket_name | last_modified | storage_class | version_id | accept_ranges | body | bucket_key_enabled | cache_control | checksum_crc32 | checksum_crc32c | checksum_sha1 | checksum_sha256 | content_disposition | content_enc>
+-----+-----+-------------+---------------+---------------+------------+---------------+------+--------------------+---------------+----------------+-----------------+---------------+-----------------+---------------------+------------>
+-----+-----+-------------+---------------+---------------+------------+---------------+------+--------------------+---------------+----------------+-----------------+---------------+-----------------+---------------------+------------>

Time: 5.1s. Rows returned: 0.

After Fix:

> select * from aws_s3_object where bucket_name = 'tes-encryption-31' and sse_customer_algorithm = 'AES256' and sse_customer_key = '/J03dxHHdcPTDNi97Aq7mYxBjnxOX0kV6UzSHVOh8es=' and sse_customer_key_md5 = 'gaWCs7+kcAeTCCLlbVdTXA=='
+-----------------+------------------------------------------------+-------------------+---------------------------+---------------+------------+---------------+-------------------------------------------------------------------------->
| key             | arn                                            | bucket_name       | last_modified             | storage_class | version_id | accept_ranges | body                                                                     >
+-----------------+------------------------------------------------+-------------------+---------------------------+---------------+------------+---------------+-------------------------------------------------------------------------->
| queryResult.txt | arn:aws:s3:::tes-encryption-31/queryResult.txt | tes-encryption-31 | 2025-02-17T10:21:48+05:30 | STANDARD      | <null>     | bytes         |  > select * from helm_release                                            >
|                 |                                                |                   |                           |               |            |               |     +---------------+-----------+---------+----------+------------------+>
|                 |                                                |                   |                           |               |            |               |     | name          | namespace | version | status   | description      |>
|                 |                                                |                   |                           |               |            |               |     +---------------+-----------+---------+----------+------------------+>
|                 |                                                |                   |                           |               |            |               |     | myapp-release | default   | 1       | deployed | Install complete |>
|                 |                                                |                   |                           |               |            |               |     |               |           |         |          |                  |>
|                 |                                                |                   |                           |               |            |               |     |               |           |         |          |                  |>

[misraved]

@ParthaI after the fix, what error do I get when I don't pass in the new optional qualifiers? How would users know that they need to pass in the sse_* optional qualifiers to the query to fix the error?

[cbruno10]

Scenarios to test after latest changes:

• If I run select * from aws_s3_object where bucket = 'test-bucket'; and in that bucket, I have 50 objects encrypted with AWS SSE, what results do I get?
• If I run select * from aws_s3_object where bucket = 'test-bucket'; and in that bucket, I have 499 objects encrypted with AWS SSE and 1 encrypted with a customer key, what results do I get?
• If I run select * from aws_s3_object where bucket = 'test-bucket' and sse_customer_algorithm = '...' and sse_customer_key = '...' and sse_customer_key_md5 = '...';, and in that bucket, I have 499 objects encrypted with AWS SSE and 1 encrypted with a customer key, what results do I get?
• If I run select * from aws_s3_object where bucket = 'test-bucket' and sse_customer_algorithm = '...' and sse_customer_key = '...' and sse_customer_key_md5 = '...';, and in that bucket, I have 50 objects encrypted a customer key, what results do I get?

[ParthaI]

@cbruno10 for:

If I run select * from aws_s3_object where bucket = 'test-bucket'; and in that bucket, I have 50 objects encrypted with AWS SSE, what results do I get?

We will get the result as expected.

> select * from aws_s3_object where bucket_name = 'tp-securityhub-finding'
+-----------------------------------------------------------+-----------------------------------------------------------------------------------------------+------------------------+---------------------------+---------------+------------+---------------+-------------->
| key                                                       | arn                                                                                           | bucket_name            | last_modified             | storage_class | version_id | accept_ranges | body         >
+-----------------------------------------------------------+-----------------------------------------------------------------------------------------------+------------------------+---------------------------+---------------+------------+---------------+-------------->
| security_hub_findings_20241106071007.json                 | arn:aws:s3:::tp-securityhub-finding/security_hub_findings_20241106071007.json                 | tp-securityhub-finding | 2024-11-06T12:40:08+05:30 | STANDARD      | <null>     | bytes         | {            >
|                                                           |                                                                                               |                        |                           |               |            |               |     "version">

---

If I run select * from aws_s3_object where bucket = 'test-bucket'; and in that bucket, I have 499 objects encrypted with AWS SSE and 1 encrypted with a customer key, what results do I get?

We will get the BadRequest/Invalidrequest error for the 1 object which is encrypted with a customer key.

> select * from aws_s3_object where bucket_name = 'tes-encryption-31'

Error: aws: operation error S3: HeadObject, https response error StatusCode: 400, RequestID: 9ZB2YDV4V5M101X9, HostID: 9qaJZaghTkvaIkVMnbT6THpIjW9zZ3E6154BFv8LGuquQ0DNx0ARrpY73PPcFzZTZWTNdLBxAbA=, api error BadRequest: Bad Request (SQLSTATE HV000)

+-----+-----+-------------+---------------+---------------+------------+---------------+------+--------------------+---------------+----------------+-----------------+---------------+-----------------+---------------------+------------------+------------------+-------->
| key | arn | bucket_name | last_modified | storage_class | version_id | accept_ranges | body | bucket_key_enabled | cache_control | checksum_crc32 | checksum_crc32c | checksum_sha1 | checksum_sha256 | content_disposition | content_encoding | content_language |

---

If I run select * from aws_s3_object where bucket = 'test-bucket' and sse_customer_algorithm = '...' and sse_customer_key = '...' and sse_customer_key_md5 = '...';, and in that bucket, I have 499 objects encrypted with AWS SSE and 1 encrypted with a customer key, what results do I get?

We will still get the error BadRequest/Invalidrequest for the 499 objects that are not encrypted with a customer key, as the parameters sse_customer_key_md5, sse_customer_key, and sse_customer_algorithm is not required for making the API call.

> select * from aws_s3_object where bucket_name = 'tes-encryption-31' and sse_customer_algorithm = 'AES256' and sse_customer_key = '/J03dxHHdcPTDNi97Aq7mYxBjnxOX0kV6UzSHVOh8es=' and sse_customer_key_md5 = 'gaWCs7+kcAeTCCLlbVdTXA=='

Error: aws: operation error S3: GetObject, https response error StatusCode: 400, RequestID: JVMSGR7VFCAVNRVD, HostID: iQr4b6flx4QtxxF8u1tL7mgYj3bY8Pt9SIVmxCVM0Jpbbx8qs/+ppb8uB94HFpoQNEW6B/VSVy8=, api error InvalidRequest: The encryption parameters are not applicable to this object. (SQLSTATE HV000)

+-----+-----+-------------+---------------+---------------+------------+---------------+------+--------------------+---------------+----------------+-----------------+---------------+-----------------+---------------------+------------------+------------------+-------->
| key | arn | bucket_name | last_modified | storage_class | version_id | accept_ranges | body | bucket_key_enabled | cache_control | checksum_crc32 | checksum_crc32c | checksum_sha1 | checksum_sha256 | content_disposition | content_encoding | content_language | content>
+-----+-----+-------------+---------------+---------------+------------+---------------+------+--------------------+---------------+----------------+-----------------+---------------+-----------------+---------------------+------------------+------------------+----

---

If I run select * from aws_s3_object where bucket = 'test-bucket' and sse_customer_algorithm = '...' and sse_customer_key = '...' and sse_customer_key_md5 = '...';, and in that bucket, I have 50 objects encrypted a customer key, what results do I get?

We will get the result as expected if all the object encrypted by using the SAME customer key otherwise we will get the Forbidden error.

> select * from aws_s3_object where bucket_name = 'tes-encryption-31' and sse_customer_algorithm = 'AES256' and sse_customer_key = '/J03dxHHdcPTDNi97Aq7mYxBjnxOX0kV6UzSHVOh8es=' and sse_customer_key_md5 = 'gaWCs7+kcAeTCCLlbVdTXA=='

Error: aws: operation error S3: HeadObject, https response error StatusCode: 403, RequestID: 6TCMB48N3FFZ14G0, HostID: 2vuCzKq8Qc096f3XbSoJKQ3Sdr2vOY7U30EhbLNqwipTb1ypxHyjEobJ78rGb4oeX/drya4U3jo10mvRj1oV5Q==, api error Forbidden: Forbidden (SQLSTATE HV000)

+-----+-----+-------------+---------------+---------------+------------+---------------+------+--------------------+---------------+----------------+-----------------+---------------+-----------------+---------------------+------------------+------------------+-------->
| key | arn | bucket_name | last_modified | storage_class | version_id | accept_ranges | body | bucket_key_enabled | cache_control | checksum_crc32 | checksum_crc32c | checksum_sha1 | checksum_sha256 | content_disposition | content_encoding | content_language | content>
+-----+-----+-------------+---------------+---------------+------------+---------------+------+--------------------+---------------+----------------+-----------------+---------------+-----------------+---------------------+------------------+------------------+-------->