Downloading the Chrome WebDriver fails with Chrome >=15 (OSOE-664)

[Piedone]

This is because WebDriverManager.Net didn't follow up with the URL of ChromeDriver packages having a different URL starting with v15: rosolko/WebDriverManager.Net#253 The linked issue also has potential solutions. The task here is to fix our issue, in the following ways:

• Add a temporary fix first that we can get in to dev and out via NuGet ASAP, most possibly this one (here's some code that shows most possibly this in a wider context; we need to use it here). Publish a new NuGet release.
• Implement a long-term fix:
  • Evaluate using Selenium Manager instead of WebDriverManager.Net: that had this issue too ([rust] Use CfT endpoints to discover chromedriver 115+ SeleniumHQ/selenium#12208), which might not be released yet.
  • Evaluate using Atata.WebDriverSetup.
  • If WebDriverManager.Net is still the best choice, then fork it and submit a fix.

Jira issue

[YevgeniyShunevych]

Hi @Piedone,

You can also consider Atata.WebDriverSetup library, which handles Chrome v115+ drivers and not only.

[Piedone]

Ah nice, I didn't know about that project, thanks for chiming in Yevgeniy. Do I understand correctly that this project has exactly the same goal as the above-mentioned WebDriverManager.Net and Selenium Manager?

[YevgeniyShunevych]

Basically, yes, but with few additions.

[Piedone]

Awesome, thanks!

[Piedone]

Not yet fully done, needs a NuGet release too.

[Piedone]

@DAud-IcI I’m getting such exceptions locally randomly which does seem to be due to the changes here, and might be the same exception from the CI build in disguise:

    OpenQA.Selenium.WebDriverException : Creating the web driver failed with the message "The process cannot access the file 'E:\ShortPath\OSOCE4\NuGetTest\test\Lombiq.OSOCE.NuGet.Tests.UI\bin\Debug\net6.0\drivers\chrome\versioncache.xml' because it is being used by another process.". This can mean that there is a leftover web driver process that you have to kill manually. Full exception: System.IO.IOException: The process cannot access the file 'E:\ShortPath\OSOCE4\NuGetTest\test\Lombiq.OSOCE.NuGet.Tests.UI\bin\Debug\net6.0\drivers\chrome\versioncache.xml' because it is being used by another process.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategy(FileStream fileStream, String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean useAsync)
   at System.Xml.XmlWriterSettings.CreateWriter(String outputFileName)
   at System.Xml.XmlWriter.Create(String outputFileName, XmlWriterSettings settings)
   at System.Xml.Linq.XDocument.Save(String fileName, SaveOptions options)
   at System.Xml.Linq.XDocument.Save(String fileName)
   at Atata.WebDriverSetup.XmlFileDriverVersionCache.GetOrAdd(String browserVersion, DateTime minimumAcceptableTimestamp, Func`2 versionResolveFunction)
   at Atata.WebDriverSetup.CachedDriverVersionCorrespondingToBrowserVersionResolver.GetDriverVersionCorrespondingToBrowserVersion(String browserVersion)
   at Atata.WebDriverSetup.DriverVersionResolver.ResolveByBrowserVersion(String version)
   at Atata.WebDriverSetup.DriverVersionResolver.ResolveCorrespondingVersion()
   at Atata.WebDriverSetup.DriverVersionResolver.ResolveCorrespondingOrLatestVersion()
   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.ResolveDriverVersion(IDriverSetupStrategy setupStrategy, String version)
   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.ExecuteSetUp()
   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.SetUp()
   at Atata.WebDriverSetup.DriverSetup.AutoSetUp(String browserName)
   at Lombiq.Tests.UI.Services.WebDriverFactory.CreateDriver[TDriver](String browserName, Func`1 driverFactory)
---- System.IO.IOException : The process cannot access the file 'E:\ShortPath\OSOCE4\NuGetTest\test\Lombiq.OSOCE.NuGet.Tests.UI\bin\Debug\net6.0\drivers\chrome\versioncache.xml' because it is being used by another process.

Also got this repeatedly:

    OpenQA.Selenium.WebDriverException : Creating the web driver failed with the message "Could not find a part of the path 'E:\ShortPath\OSOCE4\test\Lombiq.UITestingToolbox\Lombiq.Tests.UI.Samples\bin\Debug\net6.0\drivers\chrome\115.0.5790.170\dl\chromedriver-win64.zip'.". This can mean that there is a leftover web driver process that you have to kill manually. Full exception: System.IO.DirectoryNotFoundException: Could not find a part of the path 'E:\ShortPath\OSOCE4\test\Lombiq.UITestingToolbox\Lombiq.Tests.UI.Samples\bin\Debug\net6.0\drivers\chrome\115.0.5790.170\dl\chromedriver-win64.zip'.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategy(FileStream fileStream, String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize)
   at System.IO.FileStream..ctor(String path, FileMode mode)
   at Atata.WebDriverSetup.HttpRequestExecutor.DownloadFile(String url, String filePath)
   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.<>c__DisplayClass4_0.<DownloadFile>b__0()
   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.ExecuteWithRetries[TResult](Func`1 operation)
   at Atata.WebDriverSetup.ReliableHttpRequestExecutor.DownloadFile(String url, String filePath)
   at Atata.WebDriverSetup.DriverSetupExecutor.DownloadDriverIfMissing(String version, String driverDestinationDirectoryPath, String driverDestinationFileName)
   at Atata.WebDriverSetup.DriverSetupExecutor.SetUp(String version)
   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.ExecuteSetUp()
   at Atata.WebDriverSetup.DriverSetupConfigurationBuilder.SetUp()
   at Atata.WebDriverSetup.DriverSetup.AutoSetUp(String browserName)
   at Lombiq.Tests.UI.Services.WebDriverFactory.CreateDriver[TDriver](String browserName, Func`1 driverFactory) in E:\ShortPath\OSOCE4\test\Lombiq.UITestingToolbox\Lombiq.Tests.UI\Services\WebDriverFactory.cs:line 166

While both of these go away on repeated runs, it seems that there's some flakiness in the driver creation (which results in repeated runs, slowing down test execution.

I'm using VS Test Explorer, which might matter since at a quick glance over a couple of the recent OSOCE CI builds I haven't seen any flakiness.

Perhaps we should run AutoSetUp() within a lock, only once for each browser? @YevgeniyShunevych any advice on this, perhaps?

[sarahelsaig]

Could you give me more context? I couldn't reproduce this in VS2022, I've even tried running the whole Lombiq.Tests.UI.Samples multiple times with no such result.

At any rate, I've added a lock around DriverSetup.AutoSetUp(browserName);, hopefully that fixes it.

[Piedone]

I get this randomly for 0, 1, or multiple tests after running them all with this button from VS:

Or highlighting the UI test projects and clicking the button to its right.

This runs tests in a project in parallel, but separate projects serially, so it's not like two UI tests projects can step on each others' feet. This is from the NuGetTest solution (where there's one UI test project anyway), where this happens just as it does in the root solution. I didn't change anything in the code but do note that this is my TestConfiguration.json (the one recommended in the docs), i.e. there's no retry to hide tests failing only randomly:

{
  "Lombiq_Tests_UI": {
    "AgentIndex": 3,
    "TimeoutConfiguration": {
      "RetryTimeoutSeconds": 5,
      "RetryIntervalMillisecondSeconds": 300,
      "PageLoadTimeoutSeconds": 120
    },
    "OrchardCoreUITestExecutorConfiguration": {
      "MaxRetryCount": 0,
      "RetryIntervalSeconds": 0,
      "MaxParallelTests": 0
    },
    "BrowserConfiguration": {
      "Headless": true
    }
  }
}

I've now run the tests for both solutions a few times. It correctly doesn't happen for the root solution, but still does for the NuGestTest one, as expected without new NuGet packages.

[sarahelsaig]

I've now run the tests for both solutions a few times. It correctly doesn't happen for the root solution, but still does for the NuGestTest one, as expected without new NuGet packages.

Sounds like my fix does the job. I've applied it to the NuGet solution too.