An idea to simplify the Odin installation on all platforms

[eliaspekkala]

The installation problem

I'm always frustrated when trying to set up a project, and the installation has many steps, huge development dependencies (like IDEs at dozens of gigabytes), and requires troubleshooting cryptic compile errors (which might need even more package installs of which the exact version is hard to find). Some of this was true for the Odin project.

I usually give up on trying a project if I can't manage to install it in less than 15 minutes. We should aim for less than 5 minutes. Then there will be more people trying Odin and fewer people giving up on installing it.

When editing the installation steps on the website in pull request #139, I afterward thought about removing these installation steps entirely instead of following them and making the installation guide unnecessary.

Here are some ideas on two changes that might simplify and streamline the installation on all platforms and remove some development dependencies and install steps.

The installation solution

The possible solution has two parts to it:

1. Use Zig's C++ compiler to build Odin.
2. Bundle LLVM inside the Odin repo.

Zig comes pre-built in a ~50 Mb archive and contains Clang, LLD, libc, libc++, etc. That is everything Odin needs to build on all platforms except for LLVM (correct me if I'm wrong), so we bundle LLVM for all platforms. Odin already bundles LLVM for Windows.

Zig (the included tools, not the language) might provide us with the same things as now but smooth out some installation hurdles on the different platforms. If we use it as a development dependency, we might be able to remove other heavier dependencies like Visual Studio and Xcode. Zig might provide a simple cross-platform way to get what we need for Odin development.

If everything works, the installation is the same for all platforms, as follows:

• Clone Odin.
• Download Zig.
• Build Odin.

What would this mean for Windows?

• Visual Studio is no longer required for the linker and MSVC since Zig bundles the LLD linker and Mingw.
• Is the VS command prompt any longer needed? Can we use CMD or PowerShell to execute commands instead? If not, can it be installed separately from Visual Studio?

What would this mean for MacOS?

• Xcode is no longer required for the linker since Zig has the LLD linker.
• Are Xcode command-line tools any longer needed? Probably not, right? I'm not sure.
• Installing Homebrew and using it to install LLVM is no longer necessary since we bundle LLVM in the Odin repository.
• There's no need to add LLVM to $PATH since its location is known.

What would this mean for Linux?

• LLVM, Clang, libc++/g++, and Make would no longer need to be installed. There should be no need to use the distro's package manager. Maintaining install guides for dozens of distros is no longer needed.
• There's no need to add LLVM to $PATH since its location is known.

What would this mean for BSD?

• I have not thought about it yet. Does the Odin project have official support for the BSDs?

What about the build system implementation?

• build_odin.sh: The code for choosing the LLVM version and the llvm-config command are no longer needed. That means the script is massively simplified. No LLVM binaries are longer needed. The zig c++ command is compatible with Clang and LLD flags since that is what's under the hood. So a drop-in replacement?
• build.bat: Similar effects to build_odin.sh in that it would use zig c++ and LLD provided by Zig.
• Makefile: We can remove the Makefile to remove another development dependency? It does little over build_odin.sh anyway.
• build.zig: A more complicated-to-implement option might be to use a build.zig file to configure the build system instead of using build_odin.sh and build.bat. Then we could run zig build to build Odin on all platforms. Though, that requires writing Zig code. I don't know the Zig language.

What about the LLVM bundle implementation?

• What is the latest LLVM version that Odin supports that works for Windows, MacOS, and Linux? Is it version 14.0.6? Or 12.0.1?
• Do we bundle the same LLVM version for all platforms to simplify finding issues? Would this bundle be hard to create and maintain?
• Which LLVM files are needed? Not all installed ones, right? Maybe /include/llvm, /include/llvm-c, and libLLVM.so for Linux? LLVM-C.lib, LLVM-C.dll, and /llvm-c for Windows? What about MacOS? Any file size concerns? What LLVM files are the same for all platforms?
• Or do we keep the LLVM bundles in a separate repo and add them to the Odin repository as a Git submodule?

Other benefits that might materialize:

• The Odin installation for a fresh developer is easier and faster.
• The total size of Odin's development dependencies are in the range of megabytes instead of tens of gigabytes (Visual Studio at ~20 Gb and Xcode at ~10 Gb).
• Zig's built-in caching speeds up Odin's development iteration.
• Maybe we can use Zig's built-in cross-compilation capabilities to provide pre-built Odin binaries for more platforms for download? Then more can try Odin without installing anything.
• There are no installation instructions for us to maintain for every different operating system and distribution.
• Maybe Odin's CI systems would become simpler and quicker.

In closing:

Would this work? Is it beneficial for the Odin project? Is relying on Zig a good idea? Is bundling LLVM for every platform resonable? Have I missed something that breaks this whole idea? There is much to think about and maybe try. Please let me know of any issues or concerns.

[matias-eduardo]

I'm all for improving the build process and agree this would make it easier short-term, but there's an art to user-facing (as opposed to CI-style) build.bat and build.sh files, and Odin's is barely scratching the surface - it's maybe 10% there. What we need is to call The Batchman to get us closer to 100% power (jk).

In theory, the only dependencies we'd need to guarantee a working executable in the hands of a user by the end of the script is curl or wget which come preinstalled pretty much everywhere (one or the other). The only reason a user-facing build.bat file should stop is if the user explicitly tells it: "No, don't download and install this locally, I just need to change something in my environment so that you can find it the next time I call the script". That's it. Otherwise, we should be willing to bring in everything but the kitchen sink to guarantee that at least one combination of prompted choices by the users gets them a working executable.

An example of a build file that's like 50% there is mmozeiko's automatic build files found in his github, but even those are not meant to be used by, well, users.

[eliaspekkala]

I'm uncertain if we could even make a script to install the current list of dependencies; we would have trouble with Visual Studio and Xcode. But with the suggested approach, it should be possible to script the entire installation if we want. Then the installation is automatic (except we need to ask user permission at each installation step). With fewer and smaller dependencies, it becomes easier to automate. Thanks for the suggestion.

[z64]

I think it is worth bringing into this discussion that the current major effort has been adding Tilde as a compiler backend for Odin, making LLVM optional. Experimental support is in master, but it is in heavy development still.

Once stable, LLVM would still be needed for highly optimized builds, and I think only x86 is supported as of this moment. From there I believe the plan is also to work on compiling to C / C-subset, that can be compiled by any other C compiler, to add further options without needing LLVM.

Tangentially, there is also the sister compiler codin that is being worked on, which is taking the more direct approach of compiling to a C subset.

[eliaspekkala]

The suggested approach seems compatible with everything you have mentioned. It could simplify the installation when choosing the LLVM backend, and part one of the suggestion ("Use Zig's C++ compiler to build Odin.") could simplify the installation when choosing the Tilde backend (since we still require Clang and LLD for the Odin frontend). As for the Codin project, this approach does not apply. Thanks for letting me know what is in the works. I'm glad to hear that there will be alternatives to LLVM.