WiX: add custom actions to handle Clang resources

[stevapple]

The Swift compiler requires Clang resources in /usr/lib/swift/clang to use Clang builtins. The layout and content is identical to /usr/lib/clang/<clang_version>, where we point a symlink on Unix platforms. Since symlinks work differently on Windows and require elevated permission, we copied the directory when building the toolchain, but failed to make it through the installer.

A prior attempt was made in #144 using HarvestDirectory, but @compnerd pointed out that it'll add to built time, so here comes the custom actions for handling it.

Closes swiftlang/swift#60534

[compnerd]

There is a large refactoring that is currently being worked upon, lets hold off on this for a bit to get that through so we can make some progress on Swift in Swift for Windows.

[stevapple]

I’m okay if it can catch up with the 5.9 release, or else we’d take it into the 5.9 branch first.

[compnerd]

Fortunately, this is limited to the installer - so I think that we should be able to bring it into the 5.9 branch once everything is settled.

[stevapple]

Rebased upon latest changes, tested locally and ready for review @compnerd @shahmishal

[stevapple]

Ping

[stevapple]

Ping @shahmishal @compnerd

[stevapple]

Ping

[compnerd]

There are a few more invasive changes planned for this, so I would rather that we wait a little bit. BTW, can you verify that this works without elevated privileges for the normal user which may not have the SeSymbolicLinkPrivilege enabled? The installer is going to drop the need for elevated privileges and switch to per-user installs. That should make it easier to enable the Swift-in-Swift use case.

[stevapple]

@compnerd I think this is ready. I've tested locally with both per-machine (with UAC) and per-user installation (without UAC) and it works smoothly.

Since it's basically copy-and-paste, I believe there's nothing to do with SeSymbolicLinkPrivilege.

[compnerd]

@stevapple oh, of course! It doesn't need that because it is copying not creating a junction! Okay, that makes sense. BTW, what is the motivation for this again? With the latest fixes, atomics should build fine without this link which I believe is what prompted you to do this in the first place? Could you also please check that this also cleans up on uninstall?

[stevapple]

BTW, what is the motivation for this again? With the latest fixes, atomics should build fine without this link which I believe is what prompted you to do this in the first place?

This is required to bring consistency between what the Swift compiler sees and Clang sees — by exposing Clang’s internal resources to Swift. Atomics is just an example where Clang’s header is involved.

Without the piece, the Swift compiler may expand the header differently with Clang, causing mismatch when importing Clang targets. It’s also necessary for Swift driver to link against libclang_rt for profiling.

Could you also please check that this also cleans up on uninstall?

Already checked✅

[compnerd]

BTW, what is the motivation for this again? With the latest fixes, atomics should build fine without this link which I believe is what prompted you to do this in the first place?

This is required to bring consistency between what the Swift compiler sees and Clang sees — by exposing Clang’s internal resources to Swift. Atomics is just an example where Clang’s header is involved.

I think that this is going in the opposite direction. Swift sees MSVC's headers, clang should see the same. I believe that the bug in modularisation was the cause of the issue. Other than swift-atomics (which I just tested with SPM) do you have any concrete examples of failures?

Without the piece, the Swift compiler may expand the header differently with Clang, causing mismatch when importing Clang targets. It’s also necessary for Swift driver to link against libclang_rt for profiling.

I'd be curious to understand this first. I don't see this as being the case as at all. The profiling support already works properly.

https://github.com/compnerd/swift-win32/blob/main/.github/workflows/coverage.yml#L42 is a concrete example, --enable-code-coverage to SPM already works. Could you share the exact failure with profiling that you are observing?

Could you also please check that this also cleans up on uninstall?

Already checked✅

Perfect!

[stevapple]

I think that this is going in the opposite direction. Swift sees MSVC's headers, clang should see the same. I believe that the bug in modularisation was the cause of the issue. Other than swift-atomics (which I just tested with SPM) do you have any concrete examples of failures?

Both Swift and Clang can see MSVC and other global headers, but Clang has internal headers acting as shims. Usually they won't affect existing headers, but they potentially can. Atomics is just an example where Clang provides its own alternative stdatomic.h with LLVM intrinsics. Swift should see these headers since it will consume Clang modules that may rely on them.

I'd be curious to understand this first. I don't see this as being the case as at all. The profiling support already works properly.

compnerd/swift-win32@main/.github/workflows/coverage.yml#L42 is a concrete example, --enable-code-coverage to SPM already works. Could you share the exact failure with profiling that you are observing?

This is because Swift Driver's profile generation support is implemented differently on Windows than on Linux. Take a look at the highlighted lines below:

• https://github.com/apple/swift-driver/blob/688a840aa3334e462ae9f648f3d3eaac017d8299/Sources/SwiftDriver/Jobs/WindowsToolchain%2BLinkerSupport.swift#L191
• https://github.com/apple/swift-driver/blob/688a840aa3334e462ae9f648f3d3eaac017d8299/Sources/SwiftDriver/Jobs/GenericUnixToolchain%2BLinkerSupport.swift#L287-L290

On Linux, Swift Driver will find libclang_rt exactly in usr/lib/swift/clang/lib and pass the absolute path to linker. On Windows we simply hand -lclang_rt over to the linker — which is Clang itself. (In theory we can say the linked library is path/to/swift/../lib/swift/clang/lib/linux/libclang_rt.profile-x86_64.a for Linux and path/to/clang/.../lib/clang/x.y.z/lib/windows/clang_rt.profile-x86_64.lib for Windows. They're totally different paths.)

I didn't mean to judge which way is better, but it's cleat that on every other platform we assume Clang resources are accessible in usr/lib/swift/clang and we hard-code the path for every platform in the driver (https://github.com/apple/swift-driver/blob/688a840aa3334e462ae9f648f3d3eaac017d8299/Sources/SwiftDriver/Jobs/Toolchain%2BLinkerSupport.swift#L25-L37).