Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce quota limit on response data (#3492) #3494

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

remie
Copy link
Contributor

@remie remie commented Jun 16, 2021

Description

Proposed fix for "Firebase emulators should impose HTTP/2 request/response size limits" (#3492)

This PR currently does not include any tests, as I'm not entirely confident that it is a good idea to emulate / test processing of a 10MB+ response stream during test execution. Please advice if you prefer me to include such test.

Scenarios Tested

  • Normal execution of https functions (non-binary / JSON responses)
  • Execution of an https function while downloading a payload of <10MB
  • Execution of an https function while downloading a payload of >10MB

Sample Commands

N/A

@google-cla google-cla bot added the cla: yes Manual indication that this has passed CLA. label Jun 16, 2021
@samtstern
Copy link
Contributor

Since this came out of a Twitter conversation between you and @mbleigh I've assigned him as a reviewer.


// eslint-disable-next-line @typescript-eslint/no-explicit-any
res.end = function (chunk: any, encoding?: any, cb?: () => void) {
if (chunk) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another thing to consider, apart from the missing unit tests, is that the data limit is not imposed on a direct call to res.end() but only on a stream (using res.write). This is somewhat deliberate, as I would not expect anyone to directly flush 10MB+ on the response stream, but if the Google Cloud Functions production code does check the single chunk it might be useful to include it here as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about the prod behavior with res.end() but this is definitely an improvement either way!

Copy link
Contributor

@samtstern samtstern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@remie I can see why you were hesitant to add unit tests but I think one test would be pretty helpful. We can add one where the function just generates random bytes until it gets to > 10MB and ensure that the middleware works.

You don't need to worry much about the test speed, the functions runtime tests are in a separate "integration" suite which won't slow down our normal unit tests.

Let me know if you have any trouble writing the test.


// eslint-disable-next-line @typescript-eslint/no-explicit-any
res.end = function (chunk: any, encoding?: any, cb?: () => void) {
if (chunk) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about the prod behavior with res.end() but this is definitely an improvement either way!

@samtstern
Copy link
Contributor

@remie let me know if you don't have time to add a unit test, I could helpl!

@remie
Copy link
Contributor Author

remie commented Jun 25, 2021

@samtstern I've been busy with some of our product releases, I was planning to circle back to this early next week. However, feel free to start / take over, I won't feel offended 😄

@samtstern
Copy link
Contributor

@remie no rush at all just checking in! Next week sounds great.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes Manual indication that this has passed CLA.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants