Some errors when installing python interface in mac with m2 #5700
Labels
Comments
|
Thanks for your interest in LightGBM. To help you with this, we need more information. Please provide the following:
|
|
Thanks for your reply! I use the following command to compile and install LightGBM-3.3.5 git clone -b v3.3.5 https://github.com/microsoft/LightGBM.git
cd LightGBM
mkdir build
cd build
cmake ..
make
cd ../python-package/
python setup.py installThen it returns the following message: /Users/zhenweilin/opt/anaconda3/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
INFO:LightGBM:Starting to compile the library.
INFO:LightGBM:Starting to compile with CMake.
Traceback (most recent call last):
File "/Users/zhenweilin/LightGBM/python-package/setup.py", line 95, in silent_call
subprocess.check_call(cmd, stderr=log, stdout=log)
File "/Users/zhenweilin/opt/anaconda3/lib/python3.9/subprocess.py", line 373, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '_lightgbm', '-I/Users/zhenweilin/LightGBM/python-package/build_cpp', '-j4']' returned non-zero exit status 2.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/zhenweilin/LightGBM/python-package/setup.py", line 334, in <module>
setup(name='lightgbm',
File "/Users/zhenweilin/opt/anaconda3/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/Users/zhenweilin/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/Users/zhenweilin/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/Users/zhenweilin/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
self.run_command(cmd)
File "/Users/zhenweilin/opt/anaconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/Users/zhenweilin/opt/anaconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
cmd_obj.run()
File "/Users/zhenweilin/LightGBM/python-package/setup.py", line 248, in run
compile_cpp(use_mingw=self.mingw, use_gpu=self.gpu, use_cuda=self.cuda, use_mpi=self.mpi,
File "/Users/zhenweilin/LightGBM/python-package/setup.py", line 199, in compile_cpp
silent_call(["make", "_lightgbm", f"-I{build_dir}", "-j4"], raise_error=True,
File "/Users/zhenweilin/LightGBM/python-package/setup.py", line 99, in silent_call
raise Exception("\n".join((error_msg, LOG_NOTICE)))
Exception: An error has occurred while building lightgbm library file
The full version of error log was saved into /Users/zhenweilin/LightGBM_compilation.logand the message in LightGBM_compilation.log is -- The C compiler identification is AppleClang 14.0.0.14000029
-- The CXX compiler identification is AppleClang 14.0.0.14000029
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES)
-- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES)
-- Could NOT find OpenMP (missing: OpenMP_C_FOUND OpenMP_CXX_FOUND)
-- Found OpenMP_C: -Xpreprocessor -fopenmp -I/opt/homebrew/opt/libomp/include (found version "5.0")
-- Found OpenMP_CXX: -Xpreprocessor -fopenmp -I/opt/homebrew/opt/libomp/include (found version "5.0")
-- Found OpenMP: TRUE (found version "5.0")
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Success
-- Using _mm_prefetch
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Success
-- Using _mm_malloc
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/zhenweilin/LightGBM/python-package/build_cpp
[ 5%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/gbdt.cpp.o
[ 5%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/boosting.cpp.o
[ 11%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/gbdt_prediction.cpp.o
[ 11%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/gbdt_model_text.cpp.o
[ 14%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/prediction_early_stop.cpp.o
[ 17%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/bin.cpp.o
[ 20%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/config.cpp.o
/Users/zhenweilin/LightGBM/python-package/compile/src/io/bin.cpp:165:9: warning: variable 'cnt_zero' set but not used [-Wunused-but-set-variable]
int cnt_zero = 0;
^
/Users/zhenweilin/LightGBM/python-package/compile/src/io/bin.cpp:166:9: warning: variable 'right_cnt_data' set but not used [-Wunused-but-set-variable]
int right_cnt_data = 0;
^
/Users/zhenweilin/LightGBM/python-package/compile/src/io/bin.cpp:164:9: warning: variable 'left_cnt_data' set but not used [-Wunused-but-set-variable]
int left_cnt_data = 0;
^
[ 23%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/config_auto.cpp.o
[ 26%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/dataset.cpp.o
[ 29%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/dataset_loader.cpp.o
[ 32%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/file_io.cpp.o
[ 35%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/json11.cpp.o
[ 38%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/metadata.cpp.o
[ 41%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/parser.cpp.o
[ 44%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/train_share_states.cpp.o
[ 47%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/tree.cpp.o
3 warnings generated.
[ 50%] Building CXX object CMakeFiles/_lightgbm.dir/src/metric/dcg_calculator.cpp.o
[ 52%] Building CXX object CMakeFiles/_lightgbm.dir/src/metric/metric.cpp.o
[ 55%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/ifaddrs_patch.cpp.o
[ 58%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linker_topo.cpp.o
[ 61%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linkers_mpi.cpp.o
[ 64%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linkers_socket.cpp.o
[ 67%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/network.cpp.o
[ 73%] Building CXX object CMakeFiles/_lightgbm.dir/src/objective/objective_function.cpp.o
[ 73%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/cuda_tree_learner.cpp.o
[ 76%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/data_parallel_tree_learner.cpp.o
[ 79%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/feature_parallel_tree_learner.cpp.o
[ 82%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/gpu_tree_learner.cpp.o
[ 85%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/linear_tree_learner.cpp.o
[ 88%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/serial_tree_learner.cpp.o
[ 91%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/tree_learner.cpp.o
[ 94%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/voting_parallel_tree_learner.cpp.o
[ 97%] Building CXX object CMakeFiles/_lightgbm.dir/src/c_api.cpp.o
[100%] Linking CXX shared library /Users/zhenweilin/LightGBM/python-package/compile/lib_lightgbm.so
ld: warning: ignoring file /opt/homebrew/opt/libomp/lib/libomp.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
Undefined symbols for architecture x86_64:
"___kmpc_barrier", referenced from:
_.omp_outlined..14 in linear_tree_learner.cpp.o
_.omp_outlined..88 in linear_tree_learner.cpp.o
"___kmpc_dispatch_init_4", referenced from:
_.omp_outlined..34 in dataset.cpp.o
_.omp_outlined..57 in dataset.cpp.o
_.omp_outlined. in dataset_loader.cpp.o
_.omp_outlined..53 in dataset_loader.cpp.o
_.omp_outlined..78 in dataset_loader.cpp.o
_.omp_outlined..79 in dataset_loader.cpp.o
_.omp_outlined..115 in metric.cpp.o
...
"___kmpc_dispatch_next_4", referenced from:
_.omp_outlined..34 in dataset.cpp.o
_.omp_outlined..57 in dataset.cpp.o
_.omp_outlined. in dataset_loader.cpp.o
_.omp_outlined..53 in dataset_loader.cpp.o
_.omp_outlined..78 in dataset_loader.cpp.o
_.omp_outlined..79 in dataset_loader.cpp.o
_.omp_outlined..115 in metric.cpp.o
...
"___kmpc_end_master", referenced from:
_.omp_outlined. in gbdt.cpp.o
_.omp_outlined. in gbdt_model_text.cpp.o
_.omp_outlined. in bin.cpp.o
_.omp_outlined..37 in dataset.cpp.o
_.omp_outlined. in train_share_states.cpp.o
_.omp_outlined..2 in tree.cpp.o
_.omp_outlined..96 in metric.cpp.o
...
"___kmpc_end_reduce_nowait", referenced from:
_.omp_outlined..25 in metadata.cpp.o
_.omp_outlined. in metric.cpp.o
_.omp_outlined..24 in metric.cpp.o
_.omp_outlined..26 in metric.cpp.o
_.omp_outlined..28 in metric.cpp.o
_.omp_outlined..30 in metric.cpp.o
_.omp_outlined..32 in metric.cpp.o
...
"___kmpc_end_serialized_parallel", referenced from:
LightGBM::Tree::Shrinkage(double) in boosting.cpp.o
LightGBM::RF::ResetTrainingData(LightGBM::Dataset const*, LightGBM::ObjectiveFunction const*, std::__1::vector<LightGBM::Metric const*, std::__1::allocator<LightGBM::Metric const*> > const&) in boosting.cpp.o
LightGBM::RF::AddValidDataset(LightGBM::Dataset const*, std::__1::vector<LightGBM::Metric const*, std::__1::allocator<LightGBM::Metric const*> > const&) in boosting.cpp.o
LightGBM::RF::MultiplyScore(int, double) in boosting.cpp.o
LightGBM::Tree::AddBias(double) in boosting.cpp.o
LightGBM::GBDT::TrainOneIter(float const*, float const*) in gbdt.cpp.o
LightGBM::ScoreUpdater::ScoreUpdater(LightGBM::Dataset const*, int) in gbdt.cpp.o
...
"___kmpc_for_static_fini", referenced from:
_.omp_outlined. in boosting.cpp.o
_.omp_outlined..20 in boosting.cpp.o
_.omp_outlined..30 in boosting.cpp.o
_.omp_outlined..32 in boosting.cpp.o
_.omp_outlined..33 in boosting.cpp.o
_.omp_outlined..40 in boosting.cpp.o
_.omp_outlined..43 in boosting.cpp.o
...
"___kmpc_for_static_init_4", referenced from:
_.omp_outlined. in boosting.cpp.o
_.omp_outlined..20 in boosting.cpp.o
_.omp_outlined..32 in boosting.cpp.o
_.omp_outlined..33 in boosting.cpp.o
_.omp_outlined..40 in boosting.cpp.o
_.omp_outlined..43 in boosting.cpp.o
_.omp_outlined..44 in boosting.cpp.o
...
"___kmpc_for_static_init_4u", referenced from:
_.omp_outlined..118 in gbdt_model_text.cpp.o
_.omp_outlined..6 in serial_tree_learner.cpp.o
"___kmpc_for_static_init_8", referenced from:
_.omp_outlined..30 in boosting.cpp.o
_.omp_outlined..10 in gbdt.cpp.o
_.omp_outlined..19 in metadata.cpp.o
_.omp_outlined..131 in c_api.cpp.o
_.omp_outlined..132 in c_api.cpp.o
"___kmpc_fork_call", referenced from:
LightGBM::GBDT::InitPredict(int, int, bool) in boosting.cpp.o
LightGBM::Tree::Shrinkage(double) in boosting.cpp.o
LightGBM::GOSS::TrainOneIter(float const*, float const*) in boosting.cpp.o
int LightGBM::ParallelPartitionRunner<int, false>::Run<true>(int, std::__1::function<int (int, int, int, int*, int*)> const&, int*) in boosting.cpp.o
LightGBM::RF::ResetTrainingData(LightGBM::Dataset const*, LightGBM::ObjectiveFunction const*, std::__1::vector<LightGBM::Metric const*, std::__1::allocator<LightGBM::Metric const*> > const&) in boosting.cpp.o
LightGBM::RF::AddValidDataset(LightGBM::Dataset const*, std::__1::vector<LightGBM::Metric const*, std::__1::allocator<LightGBM::Metric const*> > const&) in boosting.cpp.o
LightGBM::RF::Boosting() in boosting.cpp.o
...
"___kmpc_global_thread_num", referenced from:
LightGBM::Tree::Shrinkage(double) in boosting.cpp.o
int LightGBM::ParallelPartitionRunner<int, false>::Run<true>(int, std::__1::function<int (int, int, int, int*, int*)> const&, int*) in boosting.cpp.o
LightGBM::RF::ResetTrainingData(LightGBM::Dataset const*, LightGBM::ObjectiveFunction const*, std::__1::vector<LightGBM::Metric const*, std::__1::allocator<LightGBM::Metric const*> > const&) in boosting.cpp.o
LightGBM::RF::AddValidDataset(LightGBM::Dataset const*, std::__1::vector<LightGBM::Metric const*, std::__1::allocator<LightGBM::Metric const*> > const&) in boosting.cpp.o
LightGBM::RF::MultiplyScore(int, double) in boosting.cpp.o
LightGBM::Tree::AddBias(double) in boosting.cpp.o
LightGBM::GBDT::TrainOneIter(float const*, float const*) in gbdt.cpp.o
...
"___kmpc_master", referenced from:
_.omp_outlined. in gbdt.cpp.o
_.omp_outlined. in gbdt_model_text.cpp.o
_.omp_outlined. in bin.cpp.o
_.omp_outlined..37 in dataset.cpp.o
_.omp_outlined. in train_share_states.cpp.o
_.omp_outlined..2 in tree.cpp.o
_.omp_outlined..96 in metric.cpp.o
...
"___kmpc_push_num_threads", referenced from:
int LightGBM::ParallelPartitionRunner<int, false>::Run<true>(int, std::__1::function<int (int, int, int, int*, int*)> const&, int*) in boosting.cpp.o
int LightGBM::ParallelPartitionRunner<int, false>::Run<true>(int, std::__1::function<int (int, int, int, int*, int*)> const&, int*) in gbdt.cpp.o
void LightGBM::Dataset::ConstructHistogramsInner<true, true>(std::__1::vector<signed char, std::__1::allocator<signed char> > const&, int const*, int, float const*, float const*, float*, float*, LightGBM::TrainingShareStates*, double*) const in dataset.cpp.o
void LightGBM::MultiValBinWrapper::ConstructHistograms<true, false>(int const*, int, float const*, float const*, std::__1::vector<double, LightGBM::Common::AlignmentAllocator<double, 32ul> >*, double*) in dataset.cpp.o
void LightGBM::MultiValBinWrapper::ConstructHistograms<true, true>(int const*, int, float const*, float const*, std::__1::vector<double, LightGBM::Common::AlignmentAllocator<double, 32ul> >*, double*) in dataset.cpp.o
void LightGBM::Dataset::ConstructHistogramsInner<true, false>(std::__1::vector<signed char, std::__1::allocator<signed char> > const&, int const*, int, float const*, float const*, float*, float*, LightGBM::TrainingShareStates*, double*) const in dataset.cpp.o
void LightGBM::Dataset::ConstructHistogramsInner<false, true>(std::__1::vector<signed char, std::__1::allocator<signed char> > const&, int const*, int, float const*, float const*, float*, float*, LightGBM::TrainingShareStates*, double*) const in dataset.cpp.o
...
"___kmpc_reduce_nowait", referenced from:
_.omp_outlined..25 in metadata.cpp.o
_.omp_outlined. in metric.cpp.o
_.omp_outlined..24 in metric.cpp.o
_.omp_outlined..26 in metric.cpp.o
_.omp_outlined..28 in metric.cpp.o
_.omp_outlined..30 in metric.cpp.o
_.omp_outlined..32 in metric.cpp.o
...
"___kmpc_serialized_parallel", referenced from:
LightGBM::Tree::Shrinkage(double) in boosting.cpp.o
LightGBM::RF::ResetTrainingData(LightGBM::Dataset const*, LightGBM::ObjectiveFunction const*, std::__1::vector<LightGBM::Metric const*, std::__1::allocator<LightGBM::Metric const*> > const&) in boosting.cpp.o
LightGBM::RF::AddValidDataset(LightGBM::Dataset const*, std::__1::vector<LightGBM::Metric const*, std::__1::allocator<LightGBM::Metric const*> > const&) in boosting.cpp.o
LightGBM::RF::MultiplyScore(int, double) in boosting.cpp.o
LightGBM::Tree::AddBias(double) in boosting.cpp.o
LightGBM::GBDT::TrainOneIter(float const*, float const*) in gbdt.cpp.o
LightGBM::ScoreUpdater::ScoreUpdater(LightGBM::Dataset const*, int) in gbdt.cpp.o
...
"_omp_get_max_threads", referenced from:
LightGBM::LinearTreeLearner::InitLinear(LightGBM::Dataset const*, int) in linear_tree_learner.cpp.o
void Eigen::internal::parallelize_gemm<true, Eigen::internal::gemm_functor<double, long, Eigen::internal::general_matrix_matrix_product<long, double, 0, false, double, 0, false, 0, 1>, Eigen::Block<Eigen::Matrix<double, -1, -1, 0, -1, -1> const, -1, -1, false>, Eigen::Block<Eigen::Matrix<double, -1, -1, 0, -1, -1>, -1, -1, false>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::internal::gemm_blocking_space<0, double, double, -1, -1, -1, 1, false> >, long>(Eigen::internal::gemm_functor<double, long, Eigen::internal::general_matrix_matrix_product<long, double, 0, false, double, 0, false, 0, 1>, Eigen::Block<Eigen::Matrix<double, -1, -1, 0, -1, -1> const, -1, -1, false>, Eigen::Block<Eigen::Matrix<double, -1, -1, 0, -1, -1>, -1, -1, false>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::internal::gemm_blocking_space<0, double, double, -1, -1, -1, 1, false> > const&, long, long, long, bool) in linear_tree_learner.cpp.o
void Eigen::internal::parallelize_gemm<true, Eigen::internal::gemm_functor<double, long, Eigen::internal::general_matrix_matrix_product<long, double, 0, false, double, 0, false, 0, 1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::internal::gemm_blocking_space<0, double, double, -1, -1, -1, 1, false> >, long>(Eigen::internal::gemm_functor<double, long, Eigen::internal::general_matrix_matrix_product<long, double, 0, false, double, 0, false, 0, 1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::Matrix<double, -1, -1, 0, -1, -1>, Eigen::internal::gemm_blocking_space<0, double, double, -1, -1, -1, 1, false> > const&, long, long, long, bool) in linear_tree_learner.cpp.o
"_omp_get_num_threads", referenced from:
_.omp_outlined. in gbdt.cpp.o
_.omp_outlined. in gbdt_model_text.cpp.o
_.omp_outlined. in bin.cpp.o
_.omp_outlined..37 in dataset.cpp.o
_.omp_outlined. in train_share_states.cpp.o
_.omp_outlined..2 in tree.cpp.o
_.omp_outlined..96 in metric.cpp.o
...
"_omp_get_thread_num", referenced from:
_.omp_outlined..33 in gbdt.cpp.o
_.omp_outlined..81 in dataset_loader.cpp.o
_.omp_outlined..82 in dataset_loader.cpp.o
_.omp_outlined..84 in dataset_loader.cpp.o
_.omp_outlined..107 in metric.cpp.o
_.omp_outlined..108 in metric.cpp.o
_.omp_outlined..115 in metric.cpp.o
...
"_omp_set_num_threads", referenced from:
_LGBM_DatasetCreateFromFile in c_api.cpp.o
_LGBM_DatasetCreateFromSampledColumn in c_api.cpp.o
_LGBM_DatasetCreateFromMats in c_api.cpp.o
_LGBM_DatasetCreateFromCSR in c_api.cpp.o
_LGBM_DatasetCreateFromCSRFunc in c_api.cpp.o
_LGBM_DatasetCreateFromCSC in c_api.cpp.o
_LGBM_DatasetGetSubset in c_api.cpp.o
...
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [/Users/zhenweilin/LightGBM/python-package/compile/lib_lightgbm.so] Error 1
make[2]: *** [CMakeFiles/_lightgbm.dir/all] Error 2
make[1]: *** [CMakeFiles/_lightgbm.dir/rule] Error 2
make: *** [_lightgbm] Error 2 |
|
Very helpful, thanks! Unfortunately, I personally won't be able to test solutions for this as no free service I'm aware of offers access to Mac M1/M2s and Microsoft has not responded to my requests to provide maintainers here with credits or personal Mac M1/M2 machines for testing. If you're interested in investigating this yourself (since you have access to an M2) and submitting a pull request, I suspect these discussions might be related:
You could also try running brew uninstall libomp
brew install libompIt looks to me like there should be |
|
Sorry for the very long delay in response! I believe this is now fixed. I now have an M2 mac lapatop, and I've been able to build recent versions of I'm going to close this. To follow the conversation about LightGBM publishing precompiled wheels to PyPI which support the M1/M2/M3 macs, subscribe to #5328. If you return to this and find that you are still unable to build the Python package from source on an M1/M2/M3 mac, please post here. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello, LightGBM community! I try to compile and install python interface in mac with m2. But some errors occur:
LightGBM version is
3.3.5and clang version is
14.0.0The text was updated successfully, but these errors were encountered: