Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

op-acceptance-tests output does not handle package-level test declarations #14803

Closed
teddyknox opened this issue Mar 12, 2025 · 0 comments · Fixed by ethereum-optimism/infra#225 or ethereum-optimism/infra#230
Closed

op-acceptance-tests output does not handle package-level test declarations #14803

teddyknox opened this issue Mar 12, 2025 · 0 comments · Fixed by ethereum-optimism/infra#225 or ethereum-optimism/infra#230
Assignees
@scharissis
Labels
A-devnet-sdk Area: devnet-sdk C-bug Category: bugs

Comments

@teddyknox
Copy link
Contributor

Bug Description

Unfortunately there is a bug in just acceptance-test that causes package level test configurations to not appear in the test output. Configurations like the following do not seem to works:

      - package: github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/interop

And produce output like the following (see how the total column doesn't add up):

t=2025-03-12T01:16:24+0000 lvl=info msg=NewTestRunner() targetGate=localnet workDir=/go/src/github.com/ethereum-optimism/optimism
t=2025-03-12T01:16:24+0000 lvl=info msg="nat.New: created registry and test runner"
t=2025-03-12T01:16:24+0000 lvl=info msg="Starting op-acceptor"
t=2025-03-12T01:16:24+0000 lvl=info msg="Running all tests[n.runner.RunAllTests()]..."
t=2025-03-12T01:16:24+0000 lvl=info msg="Running test" validator=TestFindRPCEndpoints
t=2025-03-12T01:16:46+0000 lvl=info msg="Running test" validator=TestSystemWrapETH
t=2025-03-12T01:17:06+0000 lvl=info msg="Running test" validator=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/interop
t=2025-03-12T01:17:07+0000 lvl=info msg="runAllTestsInPackage() found tests" package=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/interop count=5
t=2025-03-12T01:17:11+0000 lvl=info msg="Running test" validator=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/isthmus
t=2025-03-12T01:17:30+0000 lvl=info msg="runAllTestsInPackage() found tests" package=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/isthmus count=4
t=2025-03-12T01:17:35+0000 lvl=info msg="Running test" validator=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/fjord
t=2025-03-12T01:17:38+0000 lvl=info msg="runAllTestsInPackage() found tests" package=github.com/ethereum-optimism/optimism/op-acceptance-tests/tests/fjord count=2
t=2025-03-12T01:17:40+0000 lvl=info msg="Printing results..."
 Acceptance Testing Results (76.2s)                                                       
 TYPE   ID                        DURATION  TESTS  PASSED  FAILED  SKIPPED  STATUS  ERROR 
 Gate   localnet                     76.2s      -       4       0        1  ✓ pass        
 Test   ├── TestFindRPCEndpoints     22.4s      1       1       0        0  ✓ pass  <nil> 
        ├── TestSystemWrapETH        19.7s      1       0       0        1  - skip  <nil> 
        └──                           5.0s      1       1       0        0  ✓ pass  <nil> 
 TOTAL                                76.2S     5       4       0        1  ✓ PASS

Steps to Reproduce

Run the test configuration as set in #14802 using just acceptance-test.

Expected behavior

We expect there to be output table lines for each test package.

⚠️ Notice: Issues that do not include the following sections will be subject to closure:

  • Bug Description
  • Steps to Reproduce
  • Environment Information

Please ensure all required sections are filled out accurately to expedite the debugging process and improve issue resolution efficiency.
@teddyknox teddyknox added M-needs-triage Meta: this issue needs to be labelled A-devnet-sdk Area: devnet-sdk and removed M-needs-triage Meta: this issue needs to be labelled labels Mar 12, 2025
@teddyknox teddyknox added the C-bug Category: bugs label Mar 12, 2025
@teddyknox teddyknox mentioned this issue Mar 12, 2025
Merged
@scharissis scharissis mentioned this issue Mar 12, 2025
Merged
@teddyknox teddyknox moved this from Backlog to In Review in Protocol Team Mar 12, 2025
@github-project-automation github-project-automation bot moved this from In Review to Done in Protocol Team Mar 13, 2025
@teddyknox teddyknox reopened this Mar 13, 2025
@github-project-automation github-project-automation bot moved this from Done to In Progress in Protocol Team Mar 13, 2025
@scharissis scharissis mentioned this issue Mar 14, 2025
Merged
@github-project-automation github-project-automation bot moved this from In Progress to Done in Protocol Team Mar 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@scharissis scharissis

Labels
A-devnet-sdk Area: devnet-sdk C-bug Category: bugs
Projects
Protocol Team
Archived in project
Milestone
No milestone
2 participants
@scharissis @teddyknox