Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate load tests from OpenAPI Spec + Load-test Uitlity API #8422

Merged
merged 18 commits into from
Jul 16, 2024

Conversation

timolegros
Copy link
Collaborator

@timolegros timolegros commented Jul 10, 2024

Link to Issue

Closes: #8418

Description of Changes

  • Creates a script to generate k6 load test from OpenAPI Spec
  • Creates a private utility API for k6 load tests (protected via LOAD_TESTING_AUTH_TOKEN)
    • Only includes JWT creation for a random sample of users (for now).
  • Modifies config merging to use lodash deep merging.
  • Introduced a new pattern of defining default env var values for reusability.
  • Makes JWT_SECRET env var required (non-default value) in production environments.
  • Updated load-testing readme with creating test best practices (with references to k6 documentation).

Test Plan

  • Generate load tests from OpenAPI spec with pnpm generate-load-tests script in the root package.json
  • Start the app locally to verify env var configuration

Deployment Plan

  1. Set LOAD_TESTING_AUTH_TOKEN in all Heroku environments before deployment.

Other Considerations

@timolegros timolegros added the deployment plan (PRs only) requires manual infrastructure changes on release label Jul 12, 2024
@timolegros timolegros marked this pull request as ready for review July 12, 2024 16:39
@kurtisassad
Copy link
Contributor

When running the script I get (From the openAPIGenerator docker container)

[main] ERROR io.swagger.v3.parser.util.RemoteUrl - unable to read
java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.connect0(Native Method)
	at java.base/sun.nio.ch.Net.connect(Unknown Source)
	at java.base/sun.nio.ch.Net.connect(Unknown Source)
	at java.base/sun.nio.ch.NioSocketImpl.connect(Unknown Source)
	at java.base/java.net.Socket.connect(Unknown Source)
	at java.base/java.net.Socket.connect(Unknown Source)
	at java.base/sun.net.NetworkClient.doConnect(Unknown Source)
	at java.base/sun.net.www.http.HttpClient.openServer(Unknown Source)
	at java.base/sun.net.www.http.HttpClient.openServer(Unknown Source)
	at java.base/sun.net.www.http.HttpClient.<init>(Unknown Source)
	at java.base/sun.net.www.http.HttpClient.New(Unknown Source)
	at java.base/sun.net.www.http.HttpClient.New(Unknown Source)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
	at io.swagger.v3.parser.util.RemoteUrl.urlToString(RemoteUrl.java:151)
	at io.swagger.v3.parser.OpenAPIV3Parser.readContentFromLocation(OpenAPIV3Parser.java:290)
	at io.swagger.v3.parser.OpenAPIV3Parser.readLocation(OpenAPIV3Parser.java:95)
	at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:16)
	at org.openapitools.codegen.config.CodegenConfigurator.toContext(CodegenConfigurator.java:686)
	at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:744)
	at org.openapitools.codegen.cmd.Generate.execute(Generate.java:527)
	at org.openapitools.codegen.cmd.OpenApiGeneratorCommand.run(OpenApiGeneratorCommand.java:32)
	at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:66)

Anyways maybe its best to see if others are getting this issue first, or if its just me

@kurtisassad
Copy link
Contributor

Sorry, previous issue was that server wasn't started. New issue popped up after starting server in dev mode:

[main] WARN  o.o.codegen.DefaultCodegen - Parameter name not defined properly. Default to UNKNOWN_PARAMETER_NAME
[main] ERROR o.o.codegen.DefaultCodegen - Not handling class HeaderParameter {
    class Parameter {
        name: address_id
        in: null
        description: null
        required: false
        deprecated: null
        allowEmptyValue: null
        style: simple
        explode: false
        allowReserved: null
        schema: null
        examples: null
        example: null
        content: null
        $ref: null
    }
    in: header
} as Body Parameter at the moment

@kurtisassad
Copy link
Contributor

kurtisassad commented Jul 15, 2024

Ok so it threw the error, but actually wrote the script.js correctly. This PR is absolute 🔥. Will approve this

@timolegros
Copy link
Collaborator Author

Ok so it threw the error, but actually wrote the script.js correctly. This PR is absolute 🔥. Will approve this

Yes it does throw some errors and most are to be expected. They outline in their docs that this is "best effort" generation - not all standards are transferrable to k6 and also some of our setup is non-standard (i.e. adapter pattern over tRPC). The main thing is that it gets the structure right and helps as a reference to build utility functions/tests.

@timolegros timolegros merged commit 453fa50 into master Jul 16, 2024
9 checks passed
@timolegros timolegros deleted the tim/k6-composable-trpc-tests branch July 16, 2024 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployment plan (PRs only) requires manual infrastructure changes on release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Composable Load Tests and Utility API
3 participants