Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ASP.NET Core module installer should be improved further for Windows ARM64 #47115

Closed
1 task done
lextm opened this issue Mar 9, 2023 · 36 comments · Fixed by #59483
Closed
1 task done

ASP.NET Core module installer should be improved further for Windows ARM64 #47115

lextm opened this issue Mar 9, 2023 · 36 comments · Fixed by #59483
Labels
area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework enhancement This issue represents an ask for new feature or an enhancement to an existing one
Milestone

Comments

@lextm
Copy link
Contributor

lextm commented Mar 9, 2023

Is there an existing issue for this?

  • I have searched the existing issues

Is your feature request related to a problem? Please describe the problem.

The current installer (.NET 7 for example) does the following on Windows 11 ARM64,

  1. Install pure ARM64 build of the files to Program Files.
  2. Install pure ARM64 build of the files to Program Files (x86).

So, if people try to create three application pools on IIS for ARM64, x64, and x86, they can only get things running in the ARM64 pool (enableEmulationOnWinArm64 set to false and enable32BitAppOnWin64 set to false). All other pools will crash.

Describe the solution you'd like

The changes should be made are,

  1. Install x86 build to Program Files (x86), which resolves x86 application pools.
  2. Compile pure forwarders aspnetcorev2.dll and aspnetcorev2_outofprocess.dll and install them along with arm64/x64 bits to Program Files. This resolves both in-process and out-of-process modes for arm64/x64 application pools.

Note that an alternative way is to compile ARM64X builds of both aspnetcorev2.dll and aspnetcorev2_outofprocess.dll and install to Program Files. However, it raised many challenges and needs further investigation.

This should allow all three kinds of application pools to run properly and maximize compatibility.

A all-in-one pull request is currently opened, #47290.

Note that #47124 is no longer needed.

Additional context

I tried to build ASP.NET Core module with a ARM64X profile, but found quite a few hurdles (changes to compiler settings, extra defines and missing files to include). I guess that's why this hasn't been finished in .NET 7 timeline.

Hope it can be done in .NET 8 to complete Windows ARM64 support.

@dotnet-issue-labeler dotnet-issue-labeler bot added area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework untriaged labels Mar 9, 2023
@lextm
Copy link
Contributor Author

lextm commented Mar 9, 2023

Building ASP.NET Core module with a ARM64X profile seems to be unnecessary. ARM64X has the concept of "pure forwarder DLL" so that two forwarder DLLs (one for in-process and the other for out-of-process) should be enough to integrate the existing ARM64/x64 binaries with IIS on Windows 11 ARM64.

@lextm
Copy link
Contributor Author

lextm commented Mar 10, 2023

The proposed new folder structure is as below,

  • Under C:\Program Files\IIS\Asp.Net Core Module\V2

    • aspnetcorev2.dll (pure forwarder)
    • aspnetcorev2_x64.dll (from x64 machine C:\Program Files\IIS\Asp.Net Core Module\V2\aspnetcorev2.dll)
    • aspnetcorev2_arm64.dll (original aspnetcorev2.dll from ARM64 installation)
  • Under C:\Program Files\IIS\Asp.Net Core Module\V2\17.0.23030

    • aspnetcorev2_outofprocess.dll (pure forwarder)
    • aspnetcorev2_outofprocess_x64.dll (from x64 machine C:\Program Files\IIS\Asp.Net Core Module\V2\17.0.23030\aspnetcorev2_outofprocess.dll)
    • aspnetcorev2_outofprocess_arm64.dll (original aspnetcorev2_outofprocess.dll from ARM64 installation)

@mkArtakMSFT
Copy link
Member

@joeloff do you know what were the historical reasons for the current approach?
Are these limitations expected and do you think there is anything we can do about them?

@joeloff
Copy link
Member

joeloff commented Mar 14, 2023

Historically, the hosting bundle carried both x86/x64 bits and by default installed both on an x64 machine, unless the user passed additional options to select what was installed. The idea was to support multiple hosting scenarios where servers either supported dedicated pools for x64/x86 on separate machines or on the same machine. Additionally you could opt out of installing the runtime and only install ANCM. This was to support hosting environments for self-contained .NET applications.

The bundle can carry x86, x64, and arm64 content. .NET supports installing both x64/arm64 SxS, but I don't know whether ANCM currently supports that. You would need to make changes to the MSI to avoid the x64/arm64 copies stomping on each other since both would share the same location. In .NET we do this by modifying the install location so that x64 on arm64 installs to Program Files\dotnet\x64. You'd need something similar if you wanted to support 3 application pools.

It all depends on what you want your supported scenarios to be and what IIS supports as well.

@wtgodbe
Copy link
Member

wtgodbe commented Mar 14, 2023

@HaoK I believe you did the work to support arm64 - do you remember what the goal/design was for ANCM x64/arm64 side by side?

@lextm
Copy link
Contributor Author

lextm commented Mar 16, 2023

With some help from ARM64 Process Monitor, I successfully resolved the out-of-process crash.

So, by patching the current installer, it is possible to get ARM64/x64/x86 application pools working on Windows 11 ARM64 side by side properly. I created a separate repo to show the exact steps to patch the default installation,

https://github.com/lextm/ancm-arm64

In short, the steps are,

  1. Prepare AMR64X pure forwarders with build.cmd.
  2. Extract the actual ANCM x64 MSI package with WiX Toolset dark.exe.
  3. Copy forwarders to the right locations.
  4. Copy/rename ARM64/x64 files to the right locations (main files (aspnetcorev2_*.dll) in Program Files, out-of-process files (aspnetcorev2_outofprocess_*.dll) to IIS installation folder).
  5. Override the broken x86 files (not needed once Fix x86 folder contents in ARM64 Windows Server Hosting Bundle installer #47124 is accepted and merged).

Note that I have to copy out-of-process files to IIS installation folder right now, as LoadLibrary in HandlerResolver.cpp does not work very well with out-of-process pure forwarder. The function call does not load the actual ARM64/x64 library from the forwarder folder, but from system paths.

If #47290 can be accepted and merged, then the out-of-process files can be moved back to Program Files.

I tested the patch.ps1 and restore.ps1 scripts with .NET 7.0.4 and they work for me with an ASP.NET Core 7 MVC test app published as self-contained.

It is up to you how to incorporate the findings into the hosting bundle installer.

@lextm
Copy link
Contributor Author

lextm commented Mar 19, 2023

Consolidated all required changes in #47290.

@mkArtakMSFT
Copy link
Member

Thanks for additional details, @lextm. At this point we don't have enough signal to justify increasing our support matrix, as this will result in higher support cost. Hence, we're going to park this in the backlog and monitor to see how community reacts to this over time.
Having said that, I will close your PR for now. Thanks again for your effort and suggestions.

@mkArtakMSFT mkArtakMSFT added this to the Backlog milestone Mar 21, 2023
@mkArtakMSFT mkArtakMSFT added the enhancement This issue represents an ask for new feature or an enhancement to an existing one label Mar 21, 2023
@ghost
Copy link

ghost commented Mar 21, 2023

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

@lextm
Copy link
Contributor Author

lextm commented Mar 21, 2023

@mkArtakMSFT Thanks for the transparency.

Before enough needs are revealed, any users can apply the workaround I found in https://github.com/lextm/ancm-arm64 instead.

@lextm lextm changed the title ASP.NET Core module installer should be improved further ASP.NET Core module installer should be improved further for Windows ARM64 Mar 21, 2023
@justintoth
Copy link

This is very disappointing to hear... For anyone who is developing for .NET on a newer Macbook Pro instead of other inferior laptops, we have to run Windows 11 Preview for ARM in Parallels or VMWare. For simple setups some devs can get away with using IIS Express, but many of us require IIS and run into the dreaded "The current configuration only supports loading images built for a AMD64 processor architecture" error when running our ASP.NET Core applications. We were fortunate enough to find @lextm's github repo and apply his patches, but it would be great if Microsoft would reconsider supporting this scenario so that we can develop for .NET on Macbook Pro's without patching your ASP.NET Core dll's. And no, we can't develop directly on Mac, our applications use functionality that is only available on Windows such as SQL Server, Event log, Windows impersonation, and Windows Services.

@dotnet-policy-service dotnet-policy-service bot added the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Feb 6, 2024
@wtgodbe wtgodbe removed the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Feb 6, 2024
@dotnet-policy-service dotnet-policy-service bot added the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Feb 6, 2024
@wtgodbe wtgodbe removed the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Feb 13, 2024
@dotnet dotnet deleted a comment from dotnet-policy-service bot Feb 13, 2024
@cmhofmeister
Copy link

@mkArtakMSFT I hope you reconsider. As a developer I purchased a new MacBook to create cross platform apps specifically with MAUI. Many developers are moving to new MAcBooks with Windows VM to develop. Frankly I would have used VS for MAC, but that is another disappointment and the reason for using a VM. Just looking for the tools I need to do my job.

@AbandonedRickshaw
Copy link

This is a bit ridiculous. The amount of time spent commenting in this thread probably took longer than it would have taken just to implement the @lextm script fix.

I feel that this decision is more of a marketing strategy than anything developer-related. Currently I (and anyone else in this situation) have to run the @lextm patch every time asp modules get updated, which seems to be every VS Studio update and every other Windows Update.

Does MS just not value ARM development that much? It just seems odd that the effort has been put into creating ARM versions of Windows that then get hamstring'd by little issues like this.

@clovisribeiro
Copy link

clovisribeiro commented Jun 18, 2024

Thanks for additional details, @lextm. At this point we don't have enough signal to justify increasing our support matrix, as this will result in higher support cost. Hence, we're going to park this in the backlog and monitor to see how community reacts to this over time. Having said that, I will close your PR for now. Thanks again for your effort and suggestions.

Good that you mention higher costs, because I really think the cost of Windows ARM failing again will be much higher than the cost of fixing and supporting this use case. No, I don’t think this is critical, but the more apps can run unchanged in Windows ARM the better!

Just reminding that Microsoft launched the first Windows on ARM in the Surface tablet in 2012 and it completely failed to gain traction as there was not really any app that would run on that besides the few ones available on the Windows store at that time.

I believe one of the lessons learned is you do need compatibility so more apps can run on Windows ARM unchanged until the platform gains enough traction for developers/isvs to feel compelled to adapt/recompile their apps for the new platform.

Finally, I know pure ASPNET core apps can potentially be targeted to ARM with ease but there are countless apps on the market that rely on unmanaged components that are x64 only so they do need to be executed as x64.

@robinwilson16
Copy link

I had a working IIS setup on Arm64 where I had to install the x64 URL Rewrite module and Application Request Routing module as Microsoft has not yet made an Arm64 version of these modules but this worked fine until I installed the .NET 9 hosting bundle which then breaks IIS due to the application pools running x64 mode so it can't load .NET but if I change the application pools to disable the emulation mode then it fixes that crash but then I get a different crash with the rewrite module then not loading due to that being x64 so it seems it is not possible for URL rewrite and .NET 9 to coexist on the same server over IIS.

This is because Microsoft have not yet either, developed Arm64 versions of the common IIS modules, or whilst the functionality is there in the application pools to run in different modes, the .NET hosting bundle does not allow this as the community fix supplied by @lextm over a year and a half ago was not taken forward, or anything else to fix this.

I would have thought wanting URL rewriting and to be able to host .NET projects on IIS at the same time would be a pretty common requirement so it seems surprising there isn't a way and perhaps I was being a bit hasty moving over everything to Arm64 if basic support is lacking in some areas. It's a shame as Windows itself runs really well with with great battery life so it seems the hard work has already been done.

@lextm
Copy link
Contributor Author

lextm commented Dec 24, 2024

@robinwilson16 URL Rewrite module is actually available for ARM64 if you follow these steps, but ARR isn't yet.

@phillip-haydon
Copy link

When is the Hosting bundle going to be fixed?

@justintoth
Copy link

@lextm Are you planning on updating your ancm-arm64 repo to support .NET 9? Even after running your URL Rewrite module patch, I'm still getting errors when starting an app pool:

The Module DLL 'C:\Program Files\IIS\Asp.Net Core Module\V2\aspnetcorev2.dll' could not be loaded due to a configuration problem. The current configuration only supports loading images built for a AMD64 processor architecture.

@lextm
Copy link
Contributor Author

lextm commented Feb 13, 2025

@phillip-haydon the work is in progress for .NET 10 right now, #59483

@lextm
Copy link
Contributor Author

lextm commented Feb 13, 2025

@justintoth You don't seem to have a standard IIS applicationHost.config file on this machine, so the scripts I wrote cannot patch the elements properly. Please try a clean Windows machine and compare with the problematic one.

@phillip-haydon
Copy link

None of the patching worked, I solved it by installing the .net 3.1 hosting bundle then installing the .net 8/9 aspnet runtime.

@justintoth
Copy link

justintoth commented Feb 13, 2025

@phillip-haydon When you say the .NET 8/9 ASP.NET Runtime, do you mean the x64 or the Arm64 version?

@phillip-haydon
Copy link

@phillip-haydon When you say the .NET 8/9 ASP.NET Runtime, do you mean the x64 or the Arm64 version?

The arm version.

@lextm
Copy link
Contributor Author

lextm commented Feb 13, 2025

@phillip-haydon patching related issues should be reported separately to the repos under my account (acnm-arm64 or rewrite-arm64) so that I might review the details and see what might need improving. I don't think anyone should use end-of-life bits from .NET Core 3.1.

@justintoth
Copy link

@phillip-haydon Interestingly, whether I use the ARM versions of the .NET SDK / ASP.NET Core Runtime / .NET Runtime 9.0.2 or the x64 versions, I can only get the apps to run if the Enable Emulation on ARM64 app pool advanced setting is turned off. As soon as it's turned back on, the apps fail to start with the "The Module DLL 'C:\Program Files\IIS\Asp.Net Core Module\V2\aspnetcorev2.dll' could not be loaded due to a configuration problem." error. I'm happy to make some progress though so thanks for the tip, now I'm running into "SQL Server Spatial library could not be loaded" errors but I'm not sure if it's related to .NET 9 or the ARM64 emulation being off.

@lextm I do have a applicationHost.config file, there is nothing unconventional about my Windows dev setup (beyond it being ARM-based in Parallels hosted on a Macbook Pro.) Keep in mind though that the error I reported was with your IIS URL Rewrite patch, not your aspnet core patch. The aspnetcore dll patching worked fine in .NET 8 for myself and all the other devs on my team, so I believe there is an issue with it solely in .NET 9.

@lextm
Copy link
Contributor Author

lextm commented Feb 13, 2025

@justintoth I keep a Windows ARM64 VM for just the related repos (almost clean from Windows installation media), and I couldn't reproduce any issue with .NET 9 nor URL Rewrite module.

@lextm
Copy link
Contributor Author

lextm commented Feb 13, 2025

Closing this issue now, as all remaining work is just to finish review and merge of #59483

Future discussions around ancm-arm64 and rewrite-arm64 should be posted to those repos directly.

@lextm lextm closed this as completed Feb 13, 2025
@phillip-haydon
Copy link

@lextm do we re-open if it doesnt' work

@lextm
Copy link
Contributor Author

lextm commented Feb 13, 2025

@phillip-haydon the time frame to roll out the installer updates can be very long (.NET 8/9/10). I won't mind if this is reopened for status tracking, but I don't know what else you can do but wait.

You can build the installers from that revision in #59483 and test for yourself.

Learn how to build the code from this one.

@phillip-haydon
Copy link

@phillip-haydon Interestingly, whether I use the ARM versions of the .NET SDK / ASP.NET Core Runtime / .NET Runtime 9.0.2 or the x64 versions, I can only get the apps to run if the Enable Emulation on ARM64 app pool advanced setting is turned off. As soon as it's turned back on, the apps fail to start with the "The Module DLL 'C:\Program Files\IIS\Asp.Net Core Module\V2\aspnetcorev2.dll' could not be loaded due to a configuration problem." error. I'm happy to make some progress though so thanks for the tip, now I'm running into "SQL Server Spatial library could not be loaded" errors but I'm not sure if it's related to .NET 9 or the ARM64 emulation being off.

I'm not quite sure. I have a MacBook Pro M4 Max, and running parallels to do some testing on Windows + IIS.

So in parallels with Windows ARM i have IIS setup with a .net framework 4.8 site (working fine) and then needed to put a .NET 8 site on there too. However installing the hosting bundle from .NET 8 messed everything up, if it was ARM version, it prevented the framework site running, if it was x64 it caused the aspnetcorev2.dll to throw up.

So basically installing the .net core 3.1 hosting bundle gets the aspnetcorev2.dll setup with that, then framework site worked fine, and .net 8 with aspnet runtime and no hosting bundle worked fine.

@justintoth
Copy link

I was able to get everything set up and working!!! First of all, I didn't need the .NET 3.1 Hosting Bundle, and I didn't need the IIS URL Rewrite patch (which failed for me anyway.) What I did need was @lextm's ancm-arm64 patch, so big thanks for that still working on .NET 9!

Here are the basic steps I took: (some may be unnecessary...)

  1. Install .NET SDK 9.0.200 Arm64
  2. Install ASP.NET Core Runtime 9.0.2 Arm64
  3. Install Windows Hosting Bundle
  4. Install .NET Runtime 9.0.2 Arm64
  5. Install IIS URL Rewrite module
  6. Install Visual C++ Redistributable x64
  7. Patch ASP.NET Core 9.0.2 dll's in C:\Program Files\IIS\Asp.Net Core Module\V2 with ancm-arm64 powershell script after first extracting ASP.NET Core Runtime msi contents with WiX's dark.exe process
  8. Install IIS URL Rewrite module again
  9. Set app pools > advanced settings to Enable Emulation on ARM64

Using this approach was critical, because without the emulation on ARM64 then I was getting strange Microsoft.SqlServer.Types failed to load errors, and the patch was what allowed emulation on ARM64 to work. Also it's nice to not be relying on .NET Core 3.1 installers in 2025.

@lextm
Copy link
Contributor Author

lextm commented Feb 14, 2025

@justintoth Please create your own issue to track your own investigation, as that's irrelevant to this work item and also brings in unnecessary noise here.

For example, if your step 9 is needed, then all my efforts to create those patches and this work item are wasted. You didn't get everything working yet, because with all correct patches (or revised installers) application pools with or without emulation should all work flawlessly, even with URL Rewrite module.

Something was wrong with applicationHost.config on your machine, so you weren't able to apply the URL Rewrite module patch successfully. It is up to you whether that is important or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework enhancement This issue represents an ask for new feature or an enhancement to an existing one
Projects
None yet