Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data driven test with "non-formattable" values are considered differently than those with "formattable" values #4839

Closed
drieseng opened this issue Jan 30, 2025 · 1 comment

Comments

@drieseng
Copy link

Describe the bug

Data driven tests for which at least one of the value is "non-formattable" are handled differently than those where all values are "formattable".
When a data driven tests only has "formattable" values, each "row" is considered as a separate test.
When not all values are "formattable", the test method is considered as a single test with multiple results.

Steps To Reproduce

Compile and run the following test class:

using System.Globalization;

[TestClass]
public sealed class DataDrivenTests
{
    [DataTestMethod]
    [DynamicData(nameof(ConvertTo_Formattable_Data), DynamicDataSourceType.Method)]
    public void ConvertTo_Formattable(object value)
    {
    }

    [DataTestMethod]
    [DynamicData(nameof(ConvertTo_NonFormattable_Data), DynamicDataSourceType.Method)]
    public void ConvertTo_NonFormattable(object value)
    {
    }

    [DataTestMethod]
    [DynamicData(nameof(ConvertTo_FormattableAndNonFormattable_Data), DynamicDataSourceType.Method)]
    public void ConvertTo_FormattableAndNonFormattable(object value)
    {
    }

    private static IEnumerable<object[]> ConvertTo_Formattable_Data()
    {
        yield return new object[] { "UTF8" };
        yield return new object[] { "UTF32" };
    }

    private static IEnumerable<object[]> ConvertTo_NonFormattable_Data()
    {
        yield return new object[] { CultureInfo.InvariantCulture };
        yield return new object[] { CultureInfo.InvariantCulture };
    }

    private static IEnumerable<object[]> ConvertTo_FormattableAndNonFormattable_Data()
    {
        yield return new object[] { "UTF8" };
        yield return new object[] { "UTF32" };
        yield return new object[] { CultureInfo.InvariantCulture };
    }
}

Expected behavior

All three data driven test methods should be considered as multiple tests, one per "row" of test input.
This is how the ReSharper test runner reports the tests:
Image

Actual behavior

Only the ConvertTo_Formattable test method is considered to be a "real" data driven test method with multiple inputs:
Image

@drieseng drieseng changed the title Data driven test with "non-formattable" values are considered differently from those with "formattable" values Data driven test with "non-formattable" values are considered differently than those with "formattable" values Jan 30, 2025
@Evangelink
Copy link
Member

Thanks for the report, this is yet another example of why the current serialization mechanism is a wrong design... I'll close as duplicate of #1462

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants