Support using vcpkg to build dependencies on all platforms

[alexreinking]

This PR adds support for using vcpkg to acquire and build Halide's dependencies on all platforms.

It adds a top-level vcpkg.json file that explains the relationship between Halide's features and its dependencies. These features include the various LLVM target-s (which merely imply a dependency on the corresponding LLVM backend), serialization (flatbuffers), the python-bindings (pybind11), the wasm-executor (wabt), and a few meta-features:

• jit: enables LLVM targets corresponding to the host system
• target-all: enables all LLVM targets
• tests: depends on everything needed for the tests and apps
• developer: includes all other features

All of these are optional (since x86 and WebAssembly are forced), but jit and serialization are on by default.

vcpkg is intended to be an eventual replacement for FetchContent, at least on the buildbots. It will accelerate builds beyond ccache by directly restoring binary caches for our dependencies. Unlike FetchContent, it does not pollute our build with third-party CMake code. Indeed, our build has no idea at all when vcpkg is in use.

The primary drawback is that vcpkg installation happens during (or ahead of) configuration time, so there is some initial wait.

Try it!

I have provided many CMake presets to ease adoption. As long as you have VCPKG_ROOT set to a fresh clone of vcpkg, they should work. They come in two flavors:

• vcpkg: this acquires dependencies from the main vcpkg registry, but applies our own overlay, which disables building Python 3 (really!) and LLVM. The system is searched for these as usual.
• vcpkg-full: this disables the Halide overlay and attempts to build ALL dependencies.

All these presets enable the developer feature in VCPKG_MANIFEST_FEATURES, which can be overridden in the usual way.

Here are the commands you should use to try it locally:

• On Linux or Windows: cmake --preset release-vcpkg
• On macOS: cmake --preset macOS-vcpkg
• To use Visual Studio: cmake --preset win32. Here, vcpkg is implied and -vcpkg-full can be added to build LLVM.

Buildbot adjustments

We should adjust the buildbots to use vcpkg on all platforms and to set up the overlay and feature settings so we can remove the only bit of vcpkg-aware code in the build.

[alexreinking]

mullapudi2016_reorder strikes again

[abadams]

How big is the vcpkg llvm build tree these days? Last time I tried to install llvm this way, I couldn't, because it needed several hundred gigabytes of free space (which it doesn't clean up once it's done building).

[abadams]

Oh I see, so this explicitly avoids building llvm somehow. How does it get llvm in that case?

[alexreinking]

Oh I see, so this explicitly avoids building llvm somehow.

Yes, by setting cmake/vcpkg to be the package overlay. This includes "ports" for llvm and python3 that don't build anything.

How does it get llvm in that case?

The same way as any other dependency: by searching the system. Vcpkg merely inserts its paths at the beginning of the various find_* searches.

[alexreinking]

How big is the vcpkg llvm build tree these days?

Very big thanks to building both the debug and release binaries. It is possible to override this by setting set(VCPKG_BUILD_TYPE "release") in a so-called "custom triplet".

Docs:

• https://learn.microsoft.com/en-us/vcpkg/concepts/triplets
• https://learn.microsoft.com/en-us/vcpkg/users/triplets

---

Example (untested):

## ./arm64-osx-halide.cmake

# Inherit the built-in triplet
include("${VCPKG_ROOT_DIR}/triplets/arm64-osx.cmake")

# Disable python3
set(VCPKG_OVERLAY_PORTS "${CMAKE_CURRENT_LIST_DIR}/cmake/vcpkg/python3")

# Only build release LLVM
if (PORT STREQUAL "llvm")
    set(VCPKG_BUILD_TYPE "release")
endif ()

Build:

$ cmake --preset macOS-vcpkg-all -DVCPKG_TARGET_TRIPLET=$PWD/arm64-osx-halide.cmake