Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vitest extension is confused when same tests are run in Node and Browser mode #516

Closed
4 tasks done
kayahr opened this issue Oct 29, 2024 · 7 comments · Fixed by #517 or #519
Closed
4 tasks done

Vitest extension is confused when same tests are run in Node and Browser mode #516

kayahr opened this issue Oct 29, 2024 · 7 comments · Fixed by #517 or #519

Comments

@kayahr
Copy link

kayahr commented Oct 29, 2024

Describe the bug

The Vitest documentation suggests to use Vitest Workspaces to have both Node and Browser mode tests. I know they possibly meant having DIFFERENT tests for browser and node but in my opinion it perfectly makes sense to run the same tests in both environments to ensure that the project works correctly in Node and Browser and no node-specific or browser-specific features were used in the code.

It looks like the vitest vscode extension have some support for this scenario because it distinguishes between the two different test modes in the Testing panel. The test is listed twice and the workspace project name (Browser and Node) is shown next to it:

image

But when you click the play button on the Browser or Node Test in the Testing panel then always both Tests are executed (both useragents are logged in console):

image

When clicking the play button in the code panel (at the describe or it line) then both tests are even executed two times (both user agents are logged twice in console):

image

The context menu on the play buttons is also confused, it shows the same line twice, and no matter which one you use, both tests are executed:

image

I think this whole confusion is triggered because the Node and Browser Test have the same test path. Maybe it can be fixed by also looking at the workspace project name ("Node" and "Browser" in this case) to distinguish the tests from each other?

Reproduction

Output

[INFO 6:12:17 PM] [v1.6.2] Vitest extension is activated because Vitest is installed or there is a Vite/Vitest config file in the workspace.
[INFO 6:12:17 PM] [API] Running Vitest v2.1.4 (vitest-bug-2/vitest.workspace.js) with [email protected]: /usr/bin/node 
[INFO 6:12:17 PM] [API] Vitest v2.1.4 (vitest-bug-2/vitest.workspace.js) child process 489851 created
[INFO 6:12:17 PM] [VSCODE] Watching vitest-bug-2 with pattern **/*
[INFO 6:12:17 PM] [API] Collecting tests: sum.test.js
[6:12:18 PM] Not starting the runner because tests are being collected for sum.test.js
[6:12:18 PM] No test run to finish for 
[6:12:18 PM] No files to finish
[Error 6:12:18 PM] ⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯

Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.

⎯⎯⎯⎯⎯⎯ Unhandled Error ⎯⎯⎯⎯⎯⎯⎯
TypeError: Cannot read properties of undefined (reading 'supportsParallelism')
 ❯ getThreadsCount node_modules/@vitest/browser/dist/index.js:2631:55
 ❯ executeTests node_modules/@vitest/browser/dist/index.js:2544:26
 ❯ runWorkspaceTests node_modules/@vitest/browser/dist/index.js:2625:13
 ❯ Object.runTests node_modules/@vitest/browser/dist/index.js:2645:26
 ❯ node_modules/vitest/dist/chunks/resolveConfig.DPmbhVlP.js:7695:37
 ❯ process.processTicksAndRejections node:internal/process/task_queues:105:5
 ❯ executeTests node_modules/vitest/dist/chunks/resolveConfig.DPmbhVlP.js:7678:5
 ❯ node_modules/vitest/dist/chunks/cli-api.CKrRYkw8.js:10800:9
 ❯ Vitest.runFiles node_modules/vitest/dist/chunks/cli-api.CKrRYkw8.js:10820:12

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

[6:12:18 PM] There is no test run for "sum.test.js"
[6:12:18 PM] [VSCODE] Ignoring file: node_modules/.vite/vitest/results.json
[6:12:18 PM] [VSCODE] Ignoring file: sum.test.js.git
[6:12:18 PM] [VSCODE] Ignoring file: .git/FETCH_HEAD
[6:12:19 PM] [VSCODE] Ignoring file: sum.test.js.git
[INFO 6:12:21 PM] [API] Collecting tests: sum.test.js
[INFO 6:12:21 PM] [Worker] Collecting /home/k/Projects/vitest-bugs/vitest-bug-2/sum.test.js const __vite_ssr_import_0__ = await __vite_ssr_import__("/node_modules/vitest/dist/index.js", {"importedNames":["describe","it","expect"]});
const __vite_ssr_import_1__ = await __vite_ssr_import__("/sum.js", {"importedNames":["sum"]});



__vite_ssr_import_0__.describe("sum", () => {
    __vite_ssr_import_0__.it("sums numbers", () => {
        console.log("User Agent:", navigator.userAgent, "\n");
        __vite_ssr_import_0__.expect(__vite_ssr_import_1__.sum(1, 2)).toBe(3);
    });
});

//# sourceMappingSource=vite-node
//# sourceMappingURL=data:application/json;charset=utf-8;base64,eyJ2ZXJzaW9uIjozLCJzb3VyY2VzIjpbInN1bS50ZXN0LmpzIl0sInNvdXJjZXNDb250ZW50IjpbImltcG9ydCB7IGRlc2NyaWJlLCBpdCwgZXhwZWN0IH0gZnJvbSBcIi9ub2RlX21vZHVsZXMvdml0ZXN0L2Rpc3QvaW5kZXguanNcIjtcbmltcG9ydCB7IHN1bSB9IGZyb20gXCIvc3VtLmpzXCI7XG5cbmRlc2NyaWJlKFwic3VtXCIsICgpID0+IHtcbiAgICBpdChcInN1bXMgbnVtYmVyc1wiLCAoKSA9PiB7XG4gICAgICAgIGNvbnNvbGUubG9nKFwiVXNlciBBZ2VudDpcIiwgbmF2aWdhdG9yLnVzZXJBZ2VudCwgXCJcXG5cIik7XG4gICAgICAgIGV4cGVjdChzdW0oMSwgMikpLnRvQmUoMyk7XG4gICAgfSk7XG59KTtcbiJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiQUFBQSxDQUFBOztBQUEwRTtBQUM1QztBQUM5QjtBQUNBLDhCQUFRLENBQUMsQ0FBQyxHQUFHLENBQUMsQ0FBQyxDQUFDLENBQUMsQ0FBQyxDQUFDLENBQUMsQ0FBQyxDQUFDLENBQUM7QUFDdkIsQ0FBQyxDQUFDLENBQUMsQ0FBQyx3QkFBRSxDQUFDLENBQUMsSUFBSSxDQUFDLE9BQU8sQ0FBQyxDQUFDLENBQUMsQ0FBQyxDQUFDLENBQUMsQ0FBQyxDQUFDLENBQUMsQ0FBQztBQUM5QixDQUFDLENBQUMsQ0FBQyxDQUFDLENBQUMsQ0FBQyxDQUFDLENBQUMsT0FBTyxDQUFDLEdBQUcsQ0FBQyxDQUFDLElBQUksQ0FBQyxLQUFLLENBQUMsQ0FBQyxDQUFDLENBQUMsU0FBUyxDQUFDLFNBQVMsQ0FBQyxDQUFDLENBQUMsQ0FBQyxDQUFDLENBQUMsQ0FBQyxDQUFDO0FBQzlELENBQUMsQ0FBQyxDQUFDLENBQUMsQ0FBQyxDQUFDLENBQUMsQ0FBQyw0QkFBTSxDQUFDLHlCQUFHLENBQUMsQ0FBQyxDQUFDLENBQUMsQ0FBQyxDQUFDLENBQUMsQ0FBQyxJQUFJLENBQUMsQ0FBQyxDQUFDLENBQUM7QUFDbEMsQ0FBQyxDQUFDLENBQUMsQ0FBQyxDQUFDLENBQUMsQ0FBQztBQUNQLENBQUMsQ0FBQyxDQUFDOyJ9

[INFO 6:12:21 PM] [Worker] Skipping __vite_ssr_import__ (unknown call)
Skipping __vite_ssr_import__ (unknown call)
[INFO 6:12:21 PM] [Worker] Skipping sum (unknown call)
Skipping expect (unknown call)
Found it sums numbers (run)
[INFO 6:12:21 PM] [Worker] Found describe sum (run)
[INFO 6:12:21 PM] [Worker] Found location for suite sum 6:0 -> 4:0
[INFO 6:12:21 PM] [Worker] Found location for test sums numbers 7:4 -> 5:4
[6:12:25 PM] Initiating deferred test run
[INFO 6:12:25 PM] Running 1 file(s) with name pattern: ^\s?sum sums numbers$
[6:12:25 PM] The runner is starting because tests sum.test.js were started due to a file change
[6:12:25 PM] Enqueuing "sums numbers"
[6:12:25 PM] No task result for "sum.test.js", ignoring
[6:12:25 PM] No task result for "sum", ignoring
[6:12:25 PM] Enqueuing "sums numbers" because it was just collected
[6:12:25 PM] No errors found for "sum.test.js"
[6:12:25 PM] No errors found for "sum"
[6:12:25 PM] Marking "sums numbers" as passed
[6:12:26 PM] No task result for "sum.test.js", ignoring
[6:12:26 PM] No task result for "sum", ignoring
[6:12:26 PM] Enqueuing "sums numbers" because it was just collected
[6:12:26 PM] No errors found for "sum.test.js"
[6:12:26 PM] No errors found for "sum"
[6:12:26 PM] Marking "sums numbers" as passed
[6:12:26 PM] Ending test run sum.test.js
[6:12:26 PM] [VSCODE] Ignoring file: node_modules/.vite/vitest/results.json

Version

1.6.2

Validations

@sheremet-va
Copy link
Member

sheremet-va commented Oct 29, 2024

This is a limitation of Vitest API, unfortunately. It only accepts files. It behaves the same in the CLI.

I will try to find a workaround.

@kayahr
Copy link
Author

kayahr commented Oct 29, 2024

It behaves the same in the CLI.

Does it? If I run the tests with npx vitest run the tests are only executed once per project:

$ npx vitest run

 RUN  v2.1.4 /home/k/Projects/vitest-bugs/vitest-bug-2
      [Browser] Browser runner started by playwright at http://localhost:63315/

stdout | sum.test.js > sum > sums numbers
User Agent: Node.js/22 


stdout | sum.test.js > sum > sums numbers
User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/130.0.6723.31 Safari/537.36 

 ✓ |Node| sum.test.js (1)
 ✓ |Browser| sum.test.js (1)

 Test Files  2 passed (2)
      Tests  2 passed (2)
   Start at  19:20:53
   Duration  642ms (transform 52ms, setup 0ms, collect 10ms, tests 5ms, environment 0ms, prepare 366ms)

And I can run them separately without problems. npx vitest run --project Node runs the test once in node and npx vitest run --project Browser runs the same test once in browser.

@sheremet-va
Copy link
Member

The double run is weird indeed.

And I can run them separately without problems

What you cannot do in Vitest is rerun a specific test in a specific project in watch mode (which is what the extension is using). The extension never runs the vitest command.

@kayahr
Copy link
Author

kayahr commented Oct 29, 2024

What you cannot do in Vitest is rerun a specific test in a specific project in watch mode (which is what the extension is using).

Ah, ok, I understand now. So my current workaround with prefixing the top-level test description depending on test environment works because the extension and vitest itself see the same names. When the extension would add such a prefix itself internally then vitest can't find the test...

I hope you find a workaround or vitest can be extended to improve the situation. Thank you, for looking into it.

@sheremet-va
Copy link
Member

Turns out there is a non-hacky way to achieve that (#517). It doesn't work with the continuous mode yet, though.

@kayahr
Copy link
Author

kayahr commented Oct 30, 2024

@sheremet-va It's much better now, thanks a lot! But it still has some problems:

When I freshly open the project and then right-click the Test play button at the describe or it line then the extension does not show the two projects (sub menus are missing) and clicking "Run Test" only runs the Node test.

image

Clicking "Refresh Tests" in the Testing panel doesn't change that. I have to first click "Run Tests" in the Testing Panel to do a full test run and THEN the context menu show the sub menus and they work fine (Only executing the test in one project, as expected) but it is still not possible to see which one is which:

image

Maybe prefix/suffix the menu titles with the project name? Like it is done in the Testing panel?

Shall I create a new issue for this? Or do you want to reopen this one?

@sheremet-va sheremet-va reopened this Oct 30, 2024
@sheremet-va
Copy link
Member

sheremet-va commented Oct 31, 2024

Yeah, it looks like we have to go back to displaying the project as [name] in front of the test file (this time I'll put it in the back, let's see how it looks)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
2 participants