Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement more robust tests scenario using Wiremock #23

Open
Tracked by #11
jonesbusy opened this issue Oct 4, 2024 · 4 comments
Open
Tracked by #11

Implement more robust tests scenario using Wiremock #23

jonesbusy opened this issue Oct 4, 2024 · 4 comments
Labels
Milestone

Comments

@jonesbusy
Copy link
Collaborator

For example errors from registry, timeout, blob upload and retry

@jonesbusy jonesbusy mentioned this issue Oct 4, 2024
29 tasks
@jonesbusy jonesbusy added the tests label Oct 4, 2024
@jonesbusy jonesbusy added this to the 1.0.0 milestone Oct 4, 2024
@vaidikcode
Copy link
Contributor

vaidikcode commented Feb 11, 2025

@jonesbusy
I want to work on this and I have some tests in mind that utilize WireMock. Here’s a list of the tests I’m considering:

  1. Error Handling Tests
  2. Timeout Scenarios
  3. Blob Upload Testing
  4. Retry Mechanism Testing
  5. Authentication Failures (401 errors)
  6. Rate Limiting Scenarios (429 responses)
  7. Partial or Corrupted Responses
  8. Network Connectivity Issues
  9. Manifest Operations (push/pull)
  10. Concurrent Operations Testing
  11. Registry Redirect Scenarios
  12. Version Negotiation Testing

What tests do you think should be written? Once I have your feedback, I will start working on them. Thanks!

@jonesbusy
Copy link
Collaborator Author

Did you check what test are missing? Which one makes sense on your point of view? Thanks

@vaidikcode
Copy link
Contributor

We need to test shouldHandleRegistryErrors by simulating a 500 Internal Server Error. Additionally, we have not yet verified the time scenario behaviors for shouldHandleBlobUpload using WireMock. The test for shouldHandleRetry, which checks the scenario where the first call fails and the second call succeeds, has also not been implemented. Furthermore, the tests after the fifth one appear to be incomplete. However, we may not want to write tests for all of these scenarios, as adding more tests could increase the duration of the CI/CD pipeline.

@jonesbusy
Copy link
Collaborator Author

jonesbusy commented Feb 20, 2025

The duration is quiet short for now. Ideally we should also run such test in parallel

Perhaps one of the first test is to ensure

@WireMockTest
@Execution(ExecutionMode.SAME_THREAD)
public class RegistryWireMockTest {

can run in parallel

RIght now for some reason they are flaky when running in parallel (maybe due to the share Wiremock instance or worse concurrency issue on the SDK level)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants