DO NOT MERGE Upgrade llvm 18 zig 13 #6921

lukewilliamboswell · 2024-07-24T08:27:19Z

No description provided.

crates/compiler/builtins/bitcode/build.zig

flake.nix

update nixpkgs to 24.05 fixup

crates/compiler/gen_llvm/src/llvm/erased.rs

lukewilliamboswell · 2024-08-05T00:40:05Z

crates/compiler/build/src/program.rs

@@ -196,7 +196,6 @@ fn gen_from_mono_module_llvm<'a>(

    let builder = context.create_builder();
    let (dibuilder, compile_unit) = roc_gen_llvm::llvm::build::Env::new_debug_info(module);
-    let (mpm, _fpm) = roc_gen_llvm::llvm::build::construct_optimization_passes(module, opt_level);


LLVM optimisation is no longer done at function level, but only at module level.

lukewilliamboswell · 2024-08-05T00:41:27Z

crates/compiler/build/src/program.rs

+    let inkwell_opt_level = crate::target::convert_opt_level(opt_level);
+    let inkwell_llvm_passes = crate::llvm_passes::get_llvm_passes_str(opt_level);
+    let inkwell_target_machine =
+        crate::target::target_machine(target, inkwell_opt_level, inkwell::targets::RelocMode::PIC)
+            .expect("should be a valid target machine");
+
+    module
+        .run_passes(
+            inkwell_llvm_passes,
+            &inkwell_target_machine,
+            inkwell::passes::PassBuilderOptions::create(),
+        )
+        .expect("valid llvm optimization passes");


We have replaced all the locations where we use llvm optimisations with the same helpers in roc_build to get the inkwell/llvm target_machine and llvm_passes.

lukewilliamboswell · 2024-08-05T00:42:38Z

crates/compiler/build/src/target.rs

@@ -76,7 +76,7 @@ pub fn arch_str(target: Target) -> &'static str {
        roc_target::Architecture::X86_32 if cfg!(feature = "target-x86") => "x86",
        roc_target::Architecture::Aarch64 if cfg!(feature = "target-aarch64") => "aarch64",
        roc_target::Architecture::Aarch32 if cfg!(feature = "target-arm") => "arm",
-        roc_target::Architecture::Wasm32 if cfg!(feature = "target-webassembly") => "wasm32",
+        roc_target::Architecture::Wasm32 if cfg!(feature = "target-wasm32") => "wasm32",


Using the same helpers for llvm calls surfaced this issue, we were referencing a feature that doesn't exist.

lukewilliamboswell · 2024-08-05T00:43:20Z

crates/compiler/builtins/benchmark-dec.zig

+const RocStr = @import("./bitcode/src/str.zig").RocStr;
+const RocDec = @import("./bitcode/src/dec.zig").RocDec;


We moved this file, because zig 0.13.0 no longer allows relative imports from a parent/ancestor.

lukewilliamboswell · 2024-08-05T00:44:31Z

crates/compiler/builtins/benchmark-dec.zig

@@ -172,7 +184,7 @@ fn run(comptime T: type, comptime n: usize, comptime op: fn (T, T) T, v: T) u64

 // This is needed to work around a bug with using `@call` in loops.
 inline fn callWrapper(comptime T: type, call_modifier: anytype, comptime func: anytype, params: anytype) T {
-    return @call(.{ .modifier = call_modifier }, func, params);
+    return @call(call_modifier, func, params);


I wasn't sure about this change, but it seems to work just fine.

lukewilliamboswell · 2024-08-05T00:47:59Z

crates/compiler/gen_llvm/src/llvm/bitcode.rs

@@ -243,7 +243,7 @@ fn build_transform_caller_help<'a, 'ctx>(
    let block = env.builder.get_insert_block().expect("to be in a function");
    let di_location = env.builder.get_current_debug_location().unwrap();

-    let arg_type = env.context.i8_type().ptr_type(AddressSpace::default());
+    let arg_type = env.context.ptr_type(AddressSpace::default());


Since LLVM 15 pointers are not typed. Using the latest inkwell surfaces this as a warning. The fix is just to replace the ptr types with env.context.ptr_type(AddressSpace::default()) as advised by cargo check.

After making this change, there were a number of places where variables were no longer used, so I followed chippy's suggestions and removed each of these.

- // warning: use of deprecated method `inkwell::types::FunctionType::<'ctx>::ptr_type`: - // Starting from version 15.0, LLVM doesn't differentiate between pointer types. - // Use Context::ptr_type instead.

lukewilliamboswell · 2024-08-05T00:51:45Z

crates/compiler/test_gen/src/helpers/llvm.rs

-    if main_fn.verify(true) {
-        function_pass.run_on(&main_fn);
-    } else {


I don't fully understand this change. I believe LLVM optimisation no longer occurs at a function level and so the function_pass manager is no longer required. Here I have removed this, but we still keep the verification check.

Request this be looked at this more closely.

lukewilliamboswell · 2024-08-05T00:53:39Z

devtools/flake.nix

@@ -32,7 +32,7 @@
              vscodeWithExtensions = pkgs.vscode-with-extensions.override {
                vscodeExtensions = with pkgs.vscode-extensions;
                  [
-                    matklad.rust-analyzer
+                    rust-lang.rust-analyzer


matklad's rust-analyser is deprecated, nix gives us an error and forces replacement with the rust-lang version.

lukewilliamboswell · 2024-08-05T00:54:30Z

flake.nix

@@ -133,7 +137,7 @@
            1; # to run the GUI examples with NVIDIA's closed source drivers

          shellHook = ''
-            export LLVM_SYS_${llvmMajorMinorStr}_PREFIX="${llvmPkgs.llvm.dev}"
+            export LLVM_SYS_180_PREFIX="${llvmPkgs.dev}"


The LLVM MajorMinor is actually 181, but most references still want 180 so we hardcode it here.

lukewilliamboswell · 2024-08-05T02:25:39Z

crates/compiler/build/src/link.rs

-            // include libc
-            "--library",
-            "c",


This was causing issues with zig build-obj when exporting main. We had an error saying duplicate symbol.

I believe the cause of this is that now zig 0.13.0 bundles wasi-libc, and so if we include -lc we now have two main functions, one from wasi-libc for the WASI implementation because we build the host at -target wasm32-wasi and also from out host.zig.

lukewilliamboswell · 2024-08-14T21:30:15Z

@Anton-4 -- not urgent, just making a note while I'm looking at this

We need to update the following in CI workflows to use zig 13.

      - name: Update PATH to use zig 11
        run: |
          echo "PATH=/home/big-ci-user/Downloads/zig-linux-x86_64-0.11.0:$PATH" >> $GITHUB_ENV

I also think we should start using ROC_ZIG environment variable somehow, I don't see it in any of the CI scripts. Could we make this change against our current main, or would this be better in this PR?

Anton-4 · 2024-08-16T10:27:33Z

I also think we should start using ROC_ZIG environment variable somehow, I don't see it in any of the CI scripts. Could we make this change against our current main, or would this be better in this PR?

New PR on main sounds good!

This reverts commit 530cffa.

This reverts commit 0c79ccd.

lukewilliamboswell · 2024-09-05T03:12:28Z

We should merge #6859 first before this PR.

lukewilliamboswell commented Jul 24, 2024

View reviewed changes

crates/compiler/builtins/bitcode/build.zig Show resolved Hide resolved

lukewilliamboswell commented Jul 24, 2024

View reviewed changes

flake.nix Outdated Show resolved Hide resolved

lukewilliamboswell and others added 15 commits July 28, 2024 15:55

start with flake updates

01b3adf

update nixpkgs to 24.05 fixup

change nix to uze LLVM18

a42d105

back to 18, use zig-overlay and zls-overlay instead of nixpkgs

d50317d

update crates/compiler/builtins/bitcode/build.zig to zig 0.13.0

5d4f820

fixup the obivous zig errors

011c13f

fix zig build errors

a8dd624

WIP fix LLVM compiling

2ba17a7

add new zig cache to gitignore

a5d9966

WIP add script for zig platforms glue source

697bdb9

add script to copy zig glue src into platforms

2be2c02

update cli test zig platform to zig 13

1642438

remove unsued shell script

6278c4d

add ONCE call to copy zig glue files when running tests

41192b7

upgrade more test platforms to zig 13

c082025

upgrade zig bultins to zig 13

c47915a

lukewilliamboswell force-pushed the upgrade-llvm-zig branch from 409b54c to c47915a Compare July 28, 2024 06:51

lukewilliamboswell added 2 commits July 28, 2024 17:10

update reference to zig-cache

5f95b59

pin to a commit of inkwell

67d7f0a

lukewilliamboswell commented Jul 30, 2024

View reviewed changes

crates/compiler/gen_llvm/src/llvm/erased.rs Show resolved Hide resolved

lukewilliamboswell added 2 commits July 31, 2024 05:22

WIP add llvm optimisation passes

4c9610b

fix num_to_str_f32 test

a589d7d

jpospychala mentioned this pull request Jul 30, 2024

Upgrade Zig Version to 0.13.0 #6875

Open

lukewilliamboswell added 6 commits July 31, 2024 13:54

fix roc_glue test hosts

4f10552

Merge remote-tracking branch 'remote/main' into upgrade-llvm-zig

96db1bd

upgrade sort.zig to zig 13

f3b5498

try updating builder.nix

fb62425

add LLVM_SYS_180_PREFIX env

d6c6783

revert back to zig from nixpkgs, using unstable

5c1e6d4

lukewilliamboswell commented Aug 5, 2024

View reviewed changes

remove stray comment

1750e8b

lukewilliamboswell commented Aug 5, 2024

View reviewed changes

lukewilliamboswell added 2 commits August 5, 2024 11:34

restore nix/builder.nix from zls trial

16a5ed9

don't link -lc when building zig wasm hosts

0c184fb

lukewilliamboswell commented Aug 5, 2024

View reviewed changes

lukewilliamboswell changed the title ~~WIP Upgrade llvm 18 zig 13~~ Upgrade llvm 18 zig 13 Aug 14, 2024

Merge remote-tracking branch 'remote/main' into upgrade-llvm-zig

57b277d

lukewilliamboswell added 4 commits August 15, 2024 08:10

upgrade new cli test to zig 13

0a813be

fix typo

7ab1514

try new path for zig 13 in CI machine

210397a

try check for zig 13 in CI

8813a09

Merge remote-tracking branch 'remote/main' into upgrade-llvm-zig

99e2bc2

lukewilliamboswell marked this pull request as ready for review September 5, 2024 02:28

lukewilliamboswell added 2 commits September 5, 2024 12:41

use ROC_ZIG in ubuntu native CI

0c79ccd

try just two machines for CI

530cffa

lukewilliamboswell changed the title ~~Upgrade llvm 18 zig 13~~ DO NOT MERGE Upgrade llvm 18 zig 13 Sep 5, 2024

lukewilliamboswell added 2 commits September 5, 2024 13:10

Revert "try just two machines for CI"

4750dac

This reverts commit 530cffa.

Revert "use ROC_ZIG in ubuntu native CI"

60eb6e5

This reverts commit 0c79ccd.

lukewilliamboswell added the blocked Prevents this PR from auto-closing after 30 days of inactivity label Sep 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DO NOT MERGE Upgrade llvm 18 zig 13 #6921

DO NOT MERGE Upgrade llvm 18 zig 13 #6921

lukewilliamboswell commented Jul 24, 2024

lukewilliamboswell Aug 5, 2024

lukewilliamboswell Aug 5, 2024

lukewilliamboswell Aug 5, 2024

lukewilliamboswell Aug 5, 2024

lukewilliamboswell Aug 5, 2024

lukewilliamboswell Aug 5, 2024 •

edited

Loading

lukewilliamboswell Aug 5, 2024

lukewilliamboswell Aug 5, 2024

lukewilliamboswell Aug 5, 2024

lukewilliamboswell Aug 5, 2024

lukewilliamboswell commented Aug 14, 2024

Anton-4 commented Aug 16, 2024

lukewilliamboswell commented Sep 5, 2024

		const RocStr = @import("./bitcode/src/str.zig").RocStr;
		const RocDec = @import("./bitcode/src/dec.zig").RocDec;

DO NOT MERGE Upgrade llvm 18 zig 13 #6921

Are you sure you want to change the base?

DO NOT MERGE Upgrade llvm 18 zig 13 #6921

Conversation

lukewilliamboswell commented Jul 24, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lukewilliamboswell Aug 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lukewilliamboswell commented Aug 14, 2024

Anton-4 commented Aug 16, 2024

lukewilliamboswell commented Sep 5, 2024

lukewilliamboswell Aug 5, 2024 •

edited

Loading