DataRows in UWP Release builds no longer works

[dotMorten]

Describe the bug

If you use data rows in a UWP app and run the tests in release builds, you're met with the following error:

  Failed TestMethod1 (System.Object[]) [< 1 ms]
  Error Message:
   Cannot run test method 'UnitTestProject1.UnitTest1.TestMethod1': Test data doesn't match method parameters. Either the count or types are different.
Test expected 2 parameter(s), with types 'String, Int32',
but received 1 argument(s), with types 'Object[]'.

Steps To Reproduce

1. Create a new UWP Unit Test project.
2. Upgrade to v3.4.3
3. Add a datarow test

        [TestMethod]
        [DataRow("Name", 1)]
        [DataRow("Other Name", 2)]
        public void TestMethod1(string val1, int val2)
        {
            Assert.IsFalse(string.IsNullOrEmpty(val1));
            Assert.IsTrue(val2 > 0);
        }

4. Build in release mode.
5. From commandline (testexplorer doesn't seem to work with release builds) run the following command (Replace UnitTestProject1 with your appname):
   vstest.console.exe bin\x86\Release\UnitTestProject1.build.appxrecipe /InIsolation /Platform:x86 /framework:FrameworkUap10

Expected behavior

Test passes

Actual behavior

Error:

Test expected 2 parameter(s), with types 'String, Int32',
but received 1 argument(s), with types 'Object[]'.

Additional context

Also reproduced the problem with 3.4.0.
With 3.3.0 and 3.2.0 you get a different error, but along the same lines: System.ArgumentException: Object of type 'System.Object[]' cannot be converted to type 'System.String'.
v3.1.1 won't run due to adapter issue.
v3.0.4 works and can even launch the tests in release mode from test explorer

[nohwnd]

I can repro, and I can repro in VS as well.

[nohwnd]

Unchecking the

checkbox which is enabled only for Release makes it work.

[nohwnd]

Using dynamic data source does not exhibit the same problem. And grabbing the version of the DataRow attribute from 3.0.4, before it removed the additional ctors also solves the problem.

[nohwnd]

So it is this ctor, which is not present in 3.0.4. I can repro the problem by adding it to the 3.0.4 attribute code.

The data will bind to it, and then be wrapped into object[].

https://github.com/microsoft/testfx/blob/main/src/TestFramework/TestFramework/Attributes/DataSource/DataRowAttribute.cs#L28-L31

[nohwnd]

#1740 this bug, and the related change: #1878

[nohwnd]

This file for further experiments, it stops working when you uncomment the first ctor.

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Reflection;

namespace Microsoft.VisualStudio.TestTools.UnitTesting
{

    /// <summary>
    /// Attribute to define in-line data for a test method.
    /// </summary>
    [AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
    public class DataRow304Attribute : Attribute, ITestDataSource
    {
        ///// <summary>
        ///// Initializes a new instance of the <see cref="DataRowAttribute"/> class with an array of object arguments.
        ///// </summary>
        ///// <param name="data"> The data. </param>
        ///// <remarks>This constructor is only kept for CLS compliant tests.</remarks>
        //public DataRow304Attribute(object data)
        //{
        //    Data = data != null ? new[] { data } : new object[] { null };
        //}

        // ------

        /// <summary>
        /// Initializes a new instance of the <see cref="DataRow304Attribute"/> class.
        /// </summary>
        public DataRow304Attribute()
            : this(Array.Empty<object>())
        {
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="DataRow304Attribute"/> class.
        /// </summary>
        /// <param name="stringArrayData"> The string array data. </param>
        public DataRow304Attribute(string[] stringArrayData)
            : this(new object[] { stringArrayData })
        {
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="DataRow304Attribute"/> class which takes in an array of arguments.
        /// </summary>
        /// <param name="data"> The data. </param>
        public DataRow304Attribute(params object[] data)
        {
            if (data == null)
            {
                Data = new object[] { null };
            }
            else
            {
                Data = data;
            }
        }

        /// <summary>
        /// Gets data for calling test method.
        /// </summary>
        public object[] Data { get; }

        /// <summary>
        /// Gets or sets display name in test results for customization.
        /// </summary>
        public string DisplayName { get; set; }

        /// <inheritdoc />
        public IEnumerable<object[]> GetData(MethodInfo methodInfo)
        {
            return new[] { Data };
        }

        /// <inheritdoc />
        public virtual string GetDisplayName(MethodInfo methodInfo, object[] data)
        {
            if (!string.IsNullOrWhiteSpace(DisplayName))
            {
                return DisplayName;
            }

            if (data == null)
            {
                return null;
            }

            var parameters = methodInfo.GetParameters();

            // We want to force call to `data.AsEnumerable()` to ensure that objects are casted to strings (using ToString())
            // so that null do appear as "null". If you remove the call, and do string.Join(",", new object[] { null, "a" }),
            // you will get empty string while with the call you will get "null,a".
            IEnumerable<object> displayData = parameters.Length == 1 && parameters[0].ParameterType == typeof(object[])
                ? new object[] { data.AsEnumerable() }
                : data.AsEnumerable();

            return string.Format(CultureInfo.CurrentCulture, "{0} ({1})", methodInfo.Name,
                string.Join(",", displayData));
        }
    }
}

[dotMorten]

Workaround for anyone else hitting this:

#if NETFX_CORE
global using DataRowAttribute = MyNamespace.DataRowAttribute; // Requires C# LangVersion=10+
namespace MyNamespace
{
    public class DataRowAttribute : Microsoft.VisualStudio.TestTools.UnitTesting.DataRowAttribute
    {
        public DataRowAttribute(params object[] data) : base(data) { }
    }
}
#endif

[nohwnd]

We're waiting for the owner on the native toolchain to have a look.

[Evangelink]

Issue on UWP side is private but linking it here for future internal references https://devdiv.visualstudio.com/DevDiv/_workitems/edit/2135063