Support for x86_64 and arm64 builds / universal binary for macOS

[adamski]

I'm trying to modify the wscript file to support modern architectures. Here are my changes:

diff --git a/wscript b/wscript
index d4ba9ef0..8b064891 100644
--- a/wscript
+++ b/wscript
@@ -51,8 +51,8 @@ def options(ctx):
                    help='C++ standard to compile for [c++11 c++14 c++17 ...]')
 
     ctx.add_option('--arch', action='store',
-                   dest='ARCH', default="x64",
-                   help='Target architecture when compiling on OSX: i386, x64 or FAT')
+                   dest='ARCH', default="x64_64",
+                   help='Target architecture when compiling on OSX: arm64, x86_64, universal')
 
     ctx.add_option('--no-msse', action='store_true',
                    dest='NO_MSSE', default=False,
@@ -185,14 +185,18 @@ def configure(ctx):
         # the cflags properly set
         #ctx.env.CXXFLAGS += [ '-I/usr/local/include' ]
 
-        if ctx.options.ARCH == 'i386':
-            ctx.env.CXXFLAGS += ['-arch', 'i386']
-            ctx.env.LINKFLAGS += ['-arch', 'i386']
-            ctx.env.LDFLAGS = ['-arch', 'i386']
-        if ctx.options.ARCH == 'FAT':
-            ctx.env.CXXFLAGS += ['-arch', 'i386', '-arch', 'x86_64']
-            ctx.env.LINKFLAGS += ['-arch', 'i386', '-arch', 'x86_64']
-            ctx.env.LDFLAGS = ['-arch', 'i386', '-arch', 'x86_64']
+        if ctx.options.ARCH == 'arm64':
+            ctx.env.CXXFLAGS += ['-arch', 'arm64']
+            ctx.env.LINKFLAGS += ['-arch', 'arm64']
+            ctx.env.LDFLAGS = ['-arch', 'arm64']
+        if ctx.options.ARCH == 'x86_64':
+            ctx.env.CXXFLAGS += ['-arch', 'x86_64']
+            ctx.env.LINKFLAGS += ['-arch', 'x86_64']
+            ctx.env.LDFLAGS = ['-arch', 'x86_64']
+        if ctx.options.ARCH == 'universal':
+            ctx.env.CXXFLAGS += ['-arch', 'arm64', '-arch', 'x86_64']
+            ctx.env.LINKFLAGS += ['-arch', 'arm64', '-arch', 'x86_64']
+            ctx.env.LDFLAGS = ['-arch', 'arm64', '-arch', 'x86_64']

However if I run this configure command:

python3 waf configure --build-static --lightweight= --fft=KISS --arch=x86_64

I get the following warning / failed build:

[287/287] Linking build/src/libessentia.a
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive member: src/libessentia.a(nnls.c.1.o) cputype (16777228) does not match previous archive members cputype (16777223) (all members must match)
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive member: src/libessentia.a(kiss_fft.c.1.o) cputype (16777228) does not match previous archive members cputype (16777223) (all members must match)
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive member: src/libessentia.a(kiss_fftr.c.1.o) cputype (16777228) does not match previous archive members cputype (16777223) (all members must match)
warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive library: src/libessentia.a will be fat and ar(1) will not be able to operate on it
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: for architecture: x86_64 file: src/libessentia.a(essentiautil.cpp.1.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar: internal ranlib command failed

lipo reports:

Architectures in the fat file: build/src/libessentia.a are: x86_64 arm64 

I'm not sure a) how to get it to build just x86_64 (I'm on an Apple Silicon machine) or b) how to solve the ranlib command failure

[jamiebullock]

@adamski Did you ever get universal builds working on M1...?

If I build under Rosetta, I can get an x86_64 build, however any attempt to cross-compile fails.

In my case, compiling for x86_64 under an arm shell gives:

ld: warning: ignoring file src/3rdparty/nnls/nnls.c.1.o, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
Undefined symbols for architecture x86_64:
  "_nnls", referenced from:
      essentia::standard::NNLSChroma::compute() in nnlschroma.cpp.1.o
ld: symbol(s) not found for architecture x86_64

[jamiebullock]

OK... I am able to build universal with @adamski's script modifications and:

./waf configure --lightweight= --fft=ACCELERATE --arch=universal --ignore-algos=NNLSChroma

However, I then run into the same issue with ranlib ...

[293/293] Linking build/src/libessentia.a
warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive library: src/libessentia.a will be fat and ar(1) will not be able to operate on it
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: for architecture: x86_64 file: src/libessentia.a(essentiautil.cpp.1.o) has no symbols

Don't think these issues really matter though since:

1. essentiautil.cpp contains no Mac symbols anyway
2. The resulting static library works and can be linked to from a universal executable

[andresbrocco]

You should install essentia via homebrew: check out this issue.

Then, you should point your pythonpath to the installation folder, (in my case opt/homebrew/lib/python3.9/site-packages/essentia). Follow this tutorial, which advises you to create the .vscode/settings.json and launch.json files.