Device test runners for Android & iOS (xunit)

[saariniemi]

Is it possible to run xunit tests on Android and iOS?

I already have an custom built test runner but it keeps crashing under load and it is inconsistent. I want to replace this with something else. Have tried to see if there something in the github dotnet-runtime or dotnet-maui repositories that I could re-use. Found a test runner but it can't be decoupled from the source code in a simple way.

(We have multiple 1000 tests so switching framework is not really an option).

This should be a pretty common scenario but I have not been able to find anything up to date how to run xunit tests on Android.

Any suggestion how to run our existing xunit test on Android emulator?

Update 1:

XHarness was an suggestion:

I have been able to setup an maui test runner but it is still very inconsistent.

I have been looking around for different ways to use XHarness to be able to run our Unit tests written in XUnit. But I'm also open to for other solution.

I feel it is strange that there is no documentation of how you are supposed to test your backend code for mobile devices. Have not been able to find any up to date examples of how to run xUnit tests cross platform. Many of the examples are for Xamarin.

So here is a list of things that I'm investing.

1. Can I use XHarnessRunnerLibrary? But I don't understand how it supposed to be used.
   https://github.com/dotnet/runtime/tree/2350298d78a2f0f67f7815a3bcbeeaf6870687b0/src/tests/Common/XHarnessRunnerLibrary

2. Build an test runner (Maui project has an example)
   https://github.com/dotnet/maui/blob/main/src/TestUtils/samples/DeviceTests.Sample/TestUtils.DeviceTests.Sample.csproj

3. Use different test framework (NUnit seems to have Android test instrumentation)?
   https://learn.microsoft.com/en-us/xamarin/android/troubleshooting/questions/automate-android-nunit-test

4. Is remote testing with mobile devices an option?
   https://learn.microsoft.com/en-us/visualstudio/test/remote-testing?view=vs-2022

Any suggestions, documentation, blog posts would be appreciated.

[saariniemi]

I tried to decouple this but gave up after a day.

https://github.com/dotnet/maui/blob/main/src/TestUtils/samples/DeviceTests.Sample/TestUtils.DeviceTests.Sample.csproj

[MichalPetryka]

I think that the XUnit repo is a better place for this question.

[MichalStrehovsky]

Yes, it's possible and https://github.com/dotnet/xharness is what you're looking for.

[saariniemi]

XHarness was an suggestion:

I have been able to setup an maui test runner but it is still very inconsistent.

I have been looking around for different ways to use XHarness to be able to run our Unit tests written in XUnit. But I'm also open to for other solution.

I feel it is strange that there is no documentation of how you are supposed to test your backend code for mobile devices. Have not been able to find any up to date examples of how to run xUnit tests cross platform. Many of the examples are for Xamarin.

So here is a list of things that I'm investing.

1. Can I use XHarnessRunnerLibrary? But I don't understand how it supposed to be used.
   https://github.com/dotnet/runtime/tree/2350298d78a2f0f67f7815a3bcbeeaf6870687b0/src/tests/Common/XHarnessRunnerLibrary

2. Build an test runner (Maui project has an example)
   https://github.com/dotnet/maui/blob/main/src/TestUtils/samples/DeviceTests.Sample/TestUtils.DeviceTests.Sample.csproj

3. Use different test framework (NUnit seems to have Android test instrumentation)?
   https://learn.microsoft.com/en-us/xamarin/android/troubleshooting/questions/automate-android-nunit-test

4. Is remote testing with mobile devices an option?
   https://learn.microsoft.com/en-us/visualstudio/test/remote-testing?view=vs-2022

Any suggestions, documentation, blog posts would be appreciated.

[borrrden]

I am in the same situation as you, I think. I've been using XHarness successfully for this case along with this project https://github.com/shinyorg/xunit-maui (which I sadly noticed is now archived as of last month...). The trick there is re-adding the HeadlessTestRunner (e.g. https://github.com/shinyorg/xunit-maui/blob/master/xunit.runners.maui/Platforms/Android/HeadlessRunner/HeadlessTestRunner.cs)