undefined symbol: _ZN6google8Protobuf8in....... when testing test_zero_even_op.py

[liu-xb]

All the steps are ok but I met an OSError when testing test_zero_even_op.py:

OSError: /home/xbliu/densepose/densepose/build/libcaffe2_detectron_custom_ops_gpu.so: undefined symbol: _ZN6google8protobuf8internal9ArenaImpl28AllocateAlignedAndAddCleanupEmPFvPvE .

I've tried to uninstall protobuf and reinstall it but still have this error. Please kindly help me where I can correct.

By the way, will this error stop me from referring an image? Currently, I just want to estimate the human pose in an image, not training the model. Can I do this with this error? Thank you very much.

[nwsterling]

Same error here, using Anaconda python 3.5.6 and built using gcc 6.

(some_env) nick@pop-os:/c/Development/densepose$ python $DENSEPOSE/detectron/tests/test_zero_even_op.py
[E init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
Traceback (most recent call last):
File "/c/Development/densepose/detectron/tests/test_zero_even_op.py", line 117, in
c2_utils.import_custom_ops()
File "/c/Development/densepose/detectron/utils/c2.py", line 40, in import_custom_ops
dyndep.InitOpsLibrary(custom_ops_lib)
File "/home/nick/pytorch/build/caffe2/python/dyndep.py", line 35, in InitOpsLibrary
_init_impl(name)
File "/home/nick/pytorch/build/caffe2/python/dyndep.py", line 48, in _init_impl
ctypes.CDLL(path)
File "/home/nick/anaconda3/envs/some_env/lib/python3.5/ctypes/init.py", line 351, in init
self._handle = _dlopen(self._name, mode)
OSError: /c/Development/densepose/build/libcaffe2_detectron_custom_ops_gpu.so: undefined symbol: _ZN6google8protobuf8internal9ArenaImpl28AllocateAlignedAndAddCleanupEmPFvPvE

[hoangtnm]

I also also have the same problem:

$ python2 detectron/tests/test_zero_even_op.py
E0919 11:29:46.519048 18040 init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0919 11:29:46.519068 18040 init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0919 11:29:46.519085 18040 init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
Traceback (most recent call last):
File "detectron/tests/test_zero_even_op.py", line 117, in
c2_utils.import_custom_ops()
File "/home/aiteam/workspace/DensePose/detectron/utils/c2.py", line 40, in import_custom_ops
dyndep.InitOpsLibrary(custom_ops_lib)
File "/home/aiteam/workspace/pytorch/build/caffe2/python/dyndep.py", line 35, in InitOpsLibrary
_init_impl(name)
File "/home/aiteam/workspace/pytorch/build/caffe2/python/dyndep.py", line 48, in _init_impl
ctypes.CDLL(path)
File "/usr/lib/python2.7/ctypes/init.py", line 362, in init
self._handle = _dlopen(self._name, mode)
OSError: /home/aiteam/workspace/DensePose/build/libcaffe2_detectron_custom_ops_gpu.so: undefined symbol: _ZN6google8protobuf8internal9ArenaImpl28AllocateAlignedAndAddCleanupEmPFvPvE

#System:
Ubuntu 16.04
CUDA 8.0
cuDNN 6.0
Python 2.7.12

[liu-xb]

I can run the demo although I still have this error..

[vkhalidov]

This error occurs when several versions of google protocol buffers clash: one is typically embedded into PyTorch / Caffe2, the other one is the system one. Solution would be to either use the system protocol buffers when building PyTorch / Caffe2, or use the one from PyTorch / Caffe2 when building custom ops.

[ypflll]

@vkhalidov I tried to build caffe2 from source and the same error. I've turned to docker.

[roboticlemon]

@vkhalidov How can we use the protobuf from pytorch when building custom ops? What changes should I make to the build instructions?

[hiptkang]

One of simple solutions is to use protobuf from the conda installation

# you need to set CONDA_ROOT env variable.
set(PROTOBUF_LIB $ENV{CONDA_ROOT}/lib/libprotobuf.a)

If you built caffe2 from source, then

# I'm using virtualenv.
set(PROTOBUF_LIB "$ENV{VIRTUAL_ENV}/lib/python2.7/site-packages/torch/lib/libprotobuf.a")

FYI, my local version of CMakeLists.txt is as follows

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 488ea86..1efacab 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -34,7 +34,12 @@ add_library(
      caffe2_detectron_custom_ops SHARED
      ${CUSTOM_OPS_CPU_SRCS})
 
-target_link_libraries(caffe2_detectron_custom_ops caffe2_library)
+# static protobuf library
+add_library(libprotobuf STATIC IMPORTED)
+set(PROTOBUF_LIB "$ENV{VIRTUAL_ENV}/lib/python2.7/site-packages/torch/lib/libprotobuf.a")
+set_property(TARGET libprotobuf PROPERTY IMPORTED_LOCATION "${PROTOBUF_LIB}")
+
+target_link_libraries(caffe2_detectron_custom_ops caffe2_library libprotobuf)
 install(TARGETS caffe2_detectron_custom_ops DESTINATION lib)
 
 # Install custom GPU ops lib, if gpu is present.
@@ -47,6 +52,6 @@ if (CAFFE2_USE_CUDA OR CAFFE2_FOUND_CUDA)
       ${CUSTOM_OPS_CPU_SRCS}
       ${CUSTOM_OPS_GPU_SRCS})
 
-  target_link_libraries(caffe2_detectron_custom_ops_gpu caffe2_gpu_library)
+  target_link_libraries(caffe2_detectron_custom_ops_gpu caffe2_gpu_library libprotobuf)
   install(TARGETS caffe2_detectron_custom_ops_gpu DESTINATION lib)
 endif()

[roboticlemon]

@hyounsamk Thank you so much! This has it working perfectly upon rebuilding densepose.

The initial guide for compiling pytorch/caffe2 can be found here. I'm on Ubuntu 16.04, CUDA 8.0 and cuDNN 6.

For anyone reading my CMakeLists.txt is as follows for my manually built installation in ~/Documents/caffe2-pytorch

diff --git a/CMakeLists.txt b/CMakeLists.txt

-target_link_libraries(caffe2_detectron_custom_ops caffe2_library)
+# static protobuf library

+ add_library(libprotobuf STATIC IMPORTED)
+ set(PROTOBUF_LIB "/home/memesupreme/Documents/caffe2-pytorch/build/lib/libprotobuf.a")
+ set_property(TARGET libprotobuf PROPERTY IMPORTED_LOCATION "${PROTOBUF_LIB}")
+ 
+target_link_libraries(caffe2_detectron_custom_ops caffe2_library libprotobuf)


install(TARGETS caffe2_detectron_custom_ops DESTINATION lib)

# Install custom GPU ops lib, if gpu is present.
@@ -47,6 +52,6 @@ if (CAFFE2_USE_CUDA OR CAFFE2_FOUND_CUDA)
${CUSTOM_OPS_CPU_SRCS}
${CUSTOM_OPS_GPU_SRCS})

-  target_link_libraries(caffe2_detectron_custom_ops_gpu caffe2_gpu_library)
+  target_link_libraries(caffe2_detectron_custom_ops_gpu caffe2_gpu_library libprotobuf)
install(TARGETS caffe2_detectron_custom_ops_gpu DESTINATION lib)
endif()

[JustinhoCHN]

@ingramator @hyounsamk excuse me, my env location is /home/ubuntu/anaconda2/envs/densepose, then how to modify your command: set(PROTOBUF_LIB "$ENV{VIRTUAL_ENV}/lib/python2.7/site-packages/torch/lib/libprotobuf.a")?
I tried:

set(PROTOBUF_LIB "$ENV{/home/ubuntu/anaconda2/envs/densepose}/lib/python2.7/site-packages/torch/lib/libprotobuf.a")

set(PROTOBUF_LTB "$ENV/home/ubuntu/anaconda2/envs/densepose/lib/python2.7/site-packages/torch/lib/libprotobuf.a")

It'll raise "unexpected symbol error".

[JustinhoCHN]

@ingramator @hyounsamk I don't understand where to modify these command, could you please supply more details?

[JustinhoCHN]

@ingramator @hyounsamk I got this, we have to modify the CMakelist.txt under the densepose/ dir, and do some changes like yours, recompile it again, and problem solved! Thank you guys!

Notes:
If you build the caffe2 from source, you should use:

set(PROTOBUF_LIB "/home/ubuntu/projects/pytorch/build/lib/libprotobuf.a")

like @ingramator does, because {VIRTUAL_ENV}/lib/python2.7/site-packages/torch/lib/ didn't contain libprotobuf.a in my case.

[wallarug]

Great fix @ingramator ! Saved my day!

[Anikily]

@ingramator @hyounsamk I got this, we have to modify the CMakelist.txt under the densepose/ dir, and do some changes like yours, recompile it again, and problem solved! Thank you guys!

Notes:
If you build the caffe2 from source, you should use:

set(PROTOBUF_LIB "/home/ubuntu/projects/pytorch/build/lib/libprotobuf.a")

like @ingramator does, because {VIRTUAL_ENV}/lib/python2.7/site-packages/torch/lib/ didn't contain libprotobuf.a in my case.

i have the same problem have you solved it?

[ghost]

@ingramator @hyounsamk I got this, we have to modify the CMakelist.txt under the densepose/ dir, and do some changes like yours, recompile it again, and problem solved! Thank you guys!
Notes:
If you build the caffe2 from source, you should use:

set(PROTOBUF_LIB "/home/ubuntu/projects/pytorch/build/lib/libprotobuf.a")

like @ingramator does, because {VIRTUAL_ENV}/lib/python2.7/site-packages/torch/lib/ didn't contain libprotobuf.a in my case.

i have the same problem have you solved it?

+1. Same problem after add

set(PROTOBUF_LIB "/home/ubuntu/projects/pytorch/build/lib/libprotobuf.a")

[ckyleda]

How are you meant to resolve this issue if you have installed caffe via the anaconda Pytorch installation?

That version does not come with a libprotobuf.a (I've searched the entirety of my computer)

I'm beginning to think projects need to start explicitly stating which version of protobuf they are using, because nowadays nearly every problem I encounter involves a mismatched protobuf version in some way!

I have now gone through and tried several versions of protobuf:

3.5.0 and 3.5.1 are the only versions that seem to allow 'make ops' to compile.

However, neither of those versions fix this issue about an undefined symbol. There's clearly something broken in the DensePose code.

[Yilen-fan]

OSError: /home/ubuntu/densepose/build/libcaffe2_detectron_custom_ops_gpu.so: undefined symbol: _ZN6google8protobuf8internal10LogMessagelsEPKc
so,did someone meet this error , my error is not same with you
my protobuf --version is 3.6.1
so , what is the error , and what should i do ?