Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataRow execution is published as 'unspecified' with VSTest@3 task for Test plan #4738

Open
gasparnagy opened this issue Jan 20, 2025 · 0 comments

Comments

@gasparnagy
Copy link

Describe the bug

We are using the VSTest@3 task in Azure DevOps for executing the MsTest test associated to Test Cases which having data rows in the test method ([DataRow]).

Based on the issue suggestion at #992, we have disabled source discovery using [assembly: TestDataSourceDiscovery(TestDataSourceDiscoveryOption.DuringExecution)]

When we run the pipeline, it logs all data rows as executed, but in the TestRun it is reported as a single test with 'unspecified' result. (In the TRX file that is attached to the TestRun all data row execution is recorded.)

When I downgrade MSTest.TestAdapter and MSTest.TestFramework to v2.2.3 (and remove [TestDataSourceDiscovery]) it works fine.

Steps To Reproduce

  1. Create a new MsTest project.
  2. Add a test method like.
        [DataTestMethod]
        [DataRow(42)]
        [DataRow(43)]
        public void TestMethod1(int value)
        {
            Console.WriteLine($"Value: {value}");
        }
  1. Add to the project: [assembly: TestDataSourceDiscovery(TestDataSourceDiscoveryOption.DuringExecution)]
  2. Create a TestPlan, a Test Suite and a Test Case in ADO
  3. Associate the TestMethod1 to the Test Case
  4. Create a pipeline that builds the project and runs the tests based on the TestPlan/TestSuite:
- task: VSTest@3
  displayName: 'VsTest - testSuite'
  inputs:
    testSelector: testPlan
    testPlan: 552255
    testSuite: 552257
    testConfiguration: 15
    testRunTitle: 'CI1-Plan'
    platform: '$(BuildPlatform)'
    configuration: '$(BuildConfiguration)'
  1. Run pipeline and check "Tests" tab

Expected behavior

Displays the data row test as sub-tests, like with MsTest 2.2.3:

Image

Actual behavior

It is shown as 'Unspecified':

Image

Also in the related Test Run:

Image

Additional context

I have also tried with VSTest@2, made no difference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant