Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding cross-compilation support for Windows on Arm #32582

Closed
richard-townsend-arm opened this issue Mar 31, 2020 · 7 comments
Closed

Adding cross-compilation support for Windows on Arm #32582

richard-townsend-arm opened this issue Mar 31, 2020 · 7 comments
Labels
arm Issues and PRs related to the ARM platform. build Issues and PRs related to build files or the CI. windows Issues and PRs related to the Windows platform.

Comments

@richard-townsend-arm
Copy link
Contributor

richard-townsend-arm commented Mar 31, 2020

Is your feature request related to a problem? Please describe.
Follow on #25998 (Support ARM64 Windows Desktop).

The current solution to build for Windows on Arm is to build on the device itself due to host-only tools (e.g. mksnapshot) in the build process. Catch is that because Microsoft have not (yet) released a native toolchain for Windows on Arm, building Node.js is hard because you have to use an x86 toolchain running under emulation, which is slow and bumps up against 32-bit memory limitations. As a result, the Windows on Arm build has bit-rotted and doesn't currently build out of the box.

Describe the solution you'd like
To make Windows on Arm easier to support, I'd like to add cross-compilation support to node-gyp and teach Node.js' build system how to cross-compile for Arm. GYP already makes the distinction between host and target, so the idea is to generate two versions of some projects:
image

Each host target is renamed to produce a _host.exe. Each action is then re-written to call the _host.exe version, so e.g. actions which involve mksnapshot.exe are fixed up to call mksnapshot_host.exe instead. These host-only targets need to be built for x64, so we introduce a host-only node_host.sln which is built first. Then, node.sln (which contains both host and target projects) is built for ARM64. I'd also like to backport some recent V8 changes which fix support for MSVC (Microsoft's compiler) on Arm systems.

Describe alternatives you've considered
Some possible alternatives are:

  • Continue trying to build on-device connecting devices to Node.js' build infrastructure.
  • Continue trying to build on-device using a pre-built, native Clang toolchain.

The second option is quite viable, but risks introducing ABI issues for native modules and would still require changes to node-gyp.

CC @jkunkee, @refack, if you're happy with the general outline of this approach, I'll clean up my patches, get them through internal review and open some PRs.

@richardlau
Copy link
Member

cc @joaocgreis

@bnoordhuis
Copy link
Member

How do you plan to actually run mksnapshot? You're calling it host-only but it's not: it needs to run on the target to create the snapshot.

V8's native GN build has some support for cross-compiling but it does so by running mksnapshot in an emulator (qemu-user, which only works on linux.)

@richard-townsend-arm
Copy link
Contributor Author

Not sure how recently the situation changed, but V8's native mksnapshot does support cross-compilation for Arm targets. QEMU is currently used AFAICT only for MIPS-based platforms.

@jkunkee
Copy link
Contributor

jkunkee commented Apr 1, 2020

Yes, mksnapshot does support running on one platform to generate a snapshot for another platform. The limitation here is GYP's inability to support generating projects targeting the build host instead of the build target.

@joaocgreis
Copy link
Member

@richard-townsend-arm happy to see this moving forward!

We already cross-compile for Linux ARM from Linux x86. The main changes for doing that are using a compiler with support for the target (https://github.com/rvagg/rpi-newer-crosstools.git) and setting the CC and CXX variables: https://github.com/nodejs/build/blob/050fb9733697acb3dd5bb53891c2818e2c335bab/ansible/roles/cross-compiler/files/cc-selector.sh#L55-L58 . It would be good if the process for Windows was similar to that, but we'll take what we can get.

dennisameling added a commit to dennisameling/node that referenced this issue Jun 1, 2020
Adds configuration to allow building an MSI installer for Windows ARM64.
MSI can be created by running `vcbuild.bat release msi arm64`

Refs: nodejs#25998
Refs: nodejs#32582
bzoz pushed a commit that referenced this issue Jun 9, 2020
Adds configuration to allow building an MSI installer for Windows ARM64.
MSI can be created by running `vcbuild.bat release msi arm64`

Refs: #25998
Refs: #32582

PR-URL: #33689
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
codebytere pushed a commit that referenced this issue Jun 18, 2020
Adds configuration to allow building an MSI installer for Windows ARM64.
MSI can be created by running `vcbuild.bat release msi arm64`

Refs: #25998
Refs: #32582

PR-URL: #33689
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
codebytere pushed a commit that referenced this issue Jun 30, 2020
Adds configuration to allow building an MSI installer for Windows ARM64.
MSI can be created by running `vcbuild.bat release msi arm64`

Refs: #25998
Refs: #32582

PR-URL: #33689
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
codebytere pushed a commit that referenced this issue Jul 10, 2020
Adds configuration to allow building an MSI installer for Windows ARM64.
MSI can be created by running `vcbuild.bat release msi arm64`

Refs: #25998
Refs: #32582

PR-URL: #33689
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
codebytere pushed a commit that referenced this issue Jul 14, 2020
Adds configuration to allow building an MSI installer for Windows ARM64.
MSI can be created by running `vcbuild.bat release msi arm64`

Refs: #25998
Refs: #32582

PR-URL: #33689
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
dennisameling added a commit to dennisameling/node that referenced this issue Aug 8, 2020
Uses x64 node executable for running .js files in arm64
cross-compilation scenarios. MSI can now be created by
running `vcbuild.bat release msi arm64`

Refs: nodejs#25998
Refs: nodejs#32582
dennisameling added a commit to dennisameling/node that referenced this issue Aug 8, 2020
Updates the build instructions for Windows 10 ARM64, now that cross-compiling on a x64 host is possible.

Refs: nodejs#25998
Refs: nodejs#32582
joaocgreis pushed a commit that referenced this issue Aug 19, 2020
Uses x64 node executable for running .js files in arm64
cross-compilation scenarios. MSI can now be created by
running `vcbuild.bat release msi arm64`

Refs: #25998
Refs: #32582
PR-URL: #34009
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: João Reis <reis@janeasystems.com>
BethGriggs pushed a commit that referenced this issue Aug 20, 2020
Uses x64 node executable for running .js files in arm64
cross-compilation scenarios. MSI can now be created by
running `vcbuild.bat release msi arm64`

Refs: #25998
Refs: #32582
PR-URL: #34009
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: João Reis <reis@janeasystems.com>
BethGriggs pushed a commit that referenced this issue Aug 20, 2020
Uses x64 node executable for running .js files in arm64
cross-compilation scenarios. MSI can now be created by
running `vcbuild.bat release msi arm64`

Refs: #25998
Refs: #32582
PR-URL: #34009
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: João Reis <reis@janeasystems.com>
@gengjiawen
Copy link
Member

Not sure how recently the situation changed, but V8's native mksnapshot does support cross-compilation for Arm targets. QEMU is currently used AFAICT only for MIPS-based platforms.

The tricky thing is that qemu doesn't run windows. This also make testing the build hard.

@targos targos added arm Issues and PRs related to the ARM platform. windows Issues and PRs related to the Windows platform. build Issues and PRs related to build files or the CI. labels Dec 27, 2020
@StefanStojanovic
Copy link
Contributor

Cross-compilation to Windows on ARM was enabled some time ago and Windows ARM64 binaries are now a part of the official Node.js release. Closing the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arm Issues and PRs related to the ARM platform. build Issues and PRs related to build files or the CI. windows Issues and PRs related to the Windows platform.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants