LLVM Flang 17 #28

h-vetinari · 2023-07-27T10:00:41Z

Carrying forward the work from #27 to LLVM 17. Expanding the open points from the last discussion into a TODO:

Fix windows default compilation (currently failing on missing symbol)
- Looks like: error LNK2019: unresolved external symbol __udivti3 due to a combination of missing compiler-rt support when using MSVC, and probably 128bit-support.
- Currently backporting the in-progress https://reviews.llvm.org/D154660, see also [flang] Fix build with AVOID_NATIVE_UINT128_T llvm/llvm-project#63712
- Seems to have worked in conjunction with the patch and a913048
Get rid of sysroot-hacks
- Currently {{ compiler('cxx') }} in the test environment is necessary to properly find the sysroot on linux because of this; however the GCC-activation will set CONDA_BUILD_SYSROOT, which is then picked up through our patched clang.
- Solved (on linux) by Fix automatic sysroot detection clangdev-feedstock#231
libflang_main needs to depend on libfortran_main_{{ cross_platform }} (not the other way around)
- ~~we also need to put it on a path that's distinct per target triple to support e.g. having both x64 & aarch64 in the same environment (this will likely need patching of flang)~~
- Currently deprioritized, because the interaction with clang is tricky, and we need cross-compilation less urgently than windows builds.
windows activation
- we'll probably need to do something like the clang-win activation does to set +/- the equivalent of /MD, which will also likely have to go into a separate feedstock
drop osx builds
- we're can drop osx builds as soon as they become troublesome: one compiler per language & platform is enough (and in contrast to clang, which can target the gcc runtime, flang cannot target gfortran). We're still keeping linux because it's easier to debug for general problems, but will have to make it conflict with libgfortran-ng.

Potentially:

improve flang vs. compiler-rt setup (i.e. detection of the latter when using msvc; ideally avoid patching)
decide whether to carry patch renaming flang-new to flang

Also worth noting: https://reviews.llvm.org/D154869 will reshuffle the runtime libs a lot, but this didn't land for LLVM 17.

* add lit * don't build tests * adapt bld.bat * reduce parallelism in `cmake --build` * build shared libs * use cos7 sysroot * host-depend on llvm/libclang-cpp for run-exports * use flang-new for now

* use CMake modules from LLVM source * set correct path for mlir-tblgen * use relative paths in bld.bat * add mlir as a build-dep when cross-compiling * don't enable -Werror on windows * need newer glibc

* add patch to ensure compiler-rt gets linked on windows/unix directly (patching HandleCompilerRT did not work) * remove -Werror also on unix (fails on osx) * run-depend on sysroot on linux * also use -DCMAKE_MODULE_PATH on unix

* add build env for outputs; for strong run-exports and because windows expects it * add ninja to install flang * avoid clobbering libs in other outputs * remove extra call in win flang tests * remove host-deps of libfortran-main * run-depend on compiler-rt for libflang on windows * depend on conda to deduplicate already-installed files * clean-up libfortran-main

for some silly reason, conda-build insists on merging build & host envs for the later outputs, leading to resolution conflicts due to llvm-opemp vs. libgomp coming from gcc. According to the link-check on linux, it's also not used anyway...

…nda-forge-pinning 2023.07.26.21.58.02

conda-forge-webservices · 2023-07-27T10:00:45Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

…nda-forge-pinning 2023.07.27.11.28.36

h-vetinari · 2023-07-28T02:25:54Z

@isuruf, I'd like to drop libfortran_main_{{ target }} for now. The reason is that - due to the design of flang being based on the clang driver - it's being picked up in a path that belongs to clang, so developing this patch would be very painful as we'd have to build a full clang build each time.

It's also much less relevant based on the discussion that we don't need flang on linux/osx, so we don't have to worry about cross-compilation for aarch/ppc etc. The only point where I see this becoming relevant soon-ish is for win-arm64 support, but I'd like to cross that bridge when we get there, and not block flang on win-64 over that.

Finally, https://reviews.llvm.org/D154869 will overhaul a lot of the upstream integration of the runtime libs (including fortran_main), so we'd have to redo whatever patches we come up with anyway.

we don't need cross-compilation capabilities for now, linux & osx have gfortran anyway, and win-arm64 is not ready yet.

h-vetinari · 2023-08-30T15:48:46Z

Not sure why it's still necessary after your conda-forge/clangdev-feedstock#231.

That's for linux, not windows.

You had added a test using set "CONDA_BUILD_SYSROOT=" # [win] in that PR, so I thought it also applies to windows...

recipe/meta.yaml

Co-authored-by: Isuru Fernando <isuruf@gmail.com>

h-vetinari · 2023-08-31T01:15:43Z

Alright, the sysroot on linux now gets found without having {{ compiler("cxx") }} do its own activation. 🥳

h-vetinari · 2023-08-31T01:23:20Z

I've updated the todo list in the OP. The last big open question is how to do the windows compiler activation correctly. When we last spoke about this you mentioned we might need a separate feedstock. Is that necessary even if we're not doing the cross-compilation setup for now?

h-vetinari · 2023-08-31T10:24:06Z

@isuruf, what do you think we publish this to llvm_rc? That way it's hidden away from general use, but I could start testing it out in scipy, and we'll see which problems we'll run into anyway...?

isuruf · 2023-09-01T01:02:17Z

@isuruf, what do you think we publish this to llvm_rc?

Sure.

Is that necessary even if we're not doing the cross-compilation setup for now?

Nope

isuruf · 2023-09-01T02:50:35Z

decide whether to carry patch renaming flang-new to flang

Let's keep using flang-new or change to llvm-flang as Chris Lattner mentioned. I don't think it's good to name it flang just yet.

h-vetinari · 2023-09-01T04:00:47Z

OK, thanks!

Both those things (llvm_rc and flang-new) represent the state of this PR, so it sounds like we're good to merge?!

h-vetinari · 2023-09-03T07:02:20Z

Gentle ping @isuruf - OK for you if we push the button here?

isuruf · 2023-09-03T07:07:06Z

Yeah, sounds good.

h-vetinari added 20 commits July 27, 2023 20:33

spdx lint

df89a11

switch to LLVM flang

64b6169

follow upstream standalone build instructions

db120e4

switch to rc sources/target

581afc3

collected changes while building 16.0.0.rc2

2903f55

* add lit * don't build tests * adapt bld.bat * reduce parallelism in `cmake --build` * build shared libs * use cos7 sysroot * host-depend on llvm/libclang-cpp for run-exports * use flang-new for now

adapt to new lib structure

d80205c

collected changes while building 16.0.0.rc3

d78cce0

* use CMake modules from LLVM source * set correct path for mlir-tblgen * use relative paths in bld.bat * add mlir as a build-dep when cross-compiling * don't enable -Werror on windows * need newer glibc

ensure we use matching clang on osx

bc7f246

collected changes while building 16.0.0.rc4

6aff7a7

* add patch to ensure compiler-rt gets linked on windows/unix directly (patching HandleCompilerRT did not work) * remove -Werror also on unix (fails on osx) * run-depend on sysroot on linux * also use -DCMAKE_MODULE_PATH on unix

set FORTRANFLAGS in activate.sh

267195d

pull in compiler activation to test env

9d4383b

don't build shared libs on windows and osx

cc517ab

don't let libflang depend on llvm-openmp

4f035e7

for some silly reason, conda-build insists on merging build & host envs for the later outputs, leading to resolution conflicts due to llvm-opemp vs. libgomp coming from gcc. According to the link-check on linux, it's also not used anyway...

add symbol AVOID_NATIVE_UINT128_T on windows

41026ab

backport patch for AVOID_NATIVE_UINT128_T macro

f8799f5

backport https://reviews.llvm.org/D154660

92805aa

MNT: Re-rendered with conda-build 3.26.0, conda-smithy 3.24.1, and co…

44b461e

…nda-forge-pinning 2023.07.26.21.58.02

build from upstream main (in anticipation of 17.0.0-rc1)

b8dce5b

rebase patches

5e3eb68

h-vetinari added 4 commits July 28, 2023 07:50

skip osx

25f58cb

remove obsolete experimental flag

4c3e4f0

set pagefile on windows to avoid running out of space

48777a2

MNT: Re-rendered with conda-build 3.26.0, conda-smithy 3.24.1, and co…

6272906

…nda-forge-pinning 2023.07.27.11.28.36

h-vetinari mentioned this pull request Jul 28, 2023

[flang] Fix build with AVOID_NATIVE_UINT128_T llvm/llvm-project#63712

Closed

h-vetinari added 2 commits July 28, 2023 13:53

remove remnants of osx

2be106c

remove libfortran-main_{{ target_platform }}

f685a86

we don't need cross-compilation capabilities for now, linux & osx have gfortran anyway, and win-arm64 is not ready yet.

h-vetinari force-pushed the flang_17 branch from 504cdb4 to c61638c Compare July 28, 2023 02:53

h-vetinari force-pushed the flang_17 branch from 2a7ba24 to 9596d0b Compare August 30, 2023 15:49

isuruf reviewed Aug 30, 2023

View reviewed changes

recipe/meta.yaml Outdated Show resolved Hide resolved

isuruf reviewed Aug 30, 2023

View reviewed changes

recipe/meta.yaml Outdated Show resolved Hide resolved

Restrict extra compilers for test env to windows

e5812d5

Co-authored-by: Isuru Fernando <isuruf@gmail.com>

h-vetinari added 2 commits August 31, 2023 12:17

flang 17.0.0.rc3

165a2a1

try without patch for compiler-rt

fc5ddd5

h-vetinari changed the title ~~WIP: Flang 17~~ Flang 17 Aug 31, 2023

h-vetinari changed the title ~~Flang 17~~ LLVM Flang 17 Aug 31, 2023

h-vetinari marked this pull request as ready for review August 31, 2023 03:44

isuruf merged commit bfbf4bf into conda-forge:main Sep 3, 2023
4 checks passed

h-vetinari deleted the flang_17 branch September 3, 2023 07:11

This was referenced Sep 3, 2023

WIP: LLVM flang 16.0.x #27

Closed

WIP: Switch to llvm flang #25

Closed

F18 (now Flang) upstreamed #16

Closed

WIP: meson on windows (clang-cl.exe) conda-forge/scipy-feedstock#246

Closed

h-vetinari mentioned this pull request Sep 15, 2023

LLVM 17.0.0.rc3 conda-forge/clang-compiler-activation-feedstock#111

Merged

10 tasks

h-vetinari mentioned this pull request Sep 29, 2023

meson on windows with clang-cl & flang; python 3.12 support conda-forge/scipy-feedstock#255

Merged

brynpickering mentioned this pull request Dec 7, 2023

Provide cbc for Conda on Windows conda-forge/coin-or-cbc-feedstock#3

Open

1 task

This was referenced Aug 2, 2024

WIP: Create flang variant to target WASM #61

Closed

make flang a cross-compiler #63

Open

h-vetinari mentioned this pull request Sep 5, 2024

Add flang-activation conda-forge/staged-recipes#27470

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LLVM Flang 17 #28

LLVM Flang 17 #28

h-vetinari commented Jul 27, 2023 •

edited

Loading

conda-forge-webservices bot commented Jul 27, 2023

h-vetinari commented Jul 28, 2023

h-vetinari commented Aug 30, 2023

h-vetinari commented Aug 31, 2023

h-vetinari commented Aug 31, 2023

h-vetinari commented Aug 31, 2023

isuruf commented Sep 1, 2023

isuruf commented Sep 1, 2023

h-vetinari commented Sep 1, 2023

h-vetinari commented Sep 3, 2023

isuruf commented Sep 3, 2023

LLVM Flang 17 #28

LLVM Flang 17 #28

Conversation

h-vetinari commented Jul 27, 2023 • edited Loading

conda-forge-webservices bot commented Jul 27, 2023

h-vetinari commented Jul 28, 2023

h-vetinari commented Aug 30, 2023

h-vetinari commented Aug 31, 2023

h-vetinari commented Aug 31, 2023

h-vetinari commented Aug 31, 2023

isuruf commented Sep 1, 2023

isuruf commented Sep 1, 2023

h-vetinari commented Sep 1, 2023

h-vetinari commented Sep 3, 2023

isuruf commented Sep 3, 2023

h-vetinari commented Jul 27, 2023 •

edited

Loading