Auto-instrumentation for celery might segfault when mysql is loaded before

[mariojonke]

Environment

Python 3.7.5 (default, Apr 19 2020, 20:18:17) [GCC 9.2.1 20191008] on linux
Ubuntu 19.10

Auto-instrumentation might fail with a SIGSEGV on Linux with python 3.7 when running

python opentelemetry-instrumentation/src/opentelemetry/instrumentation/sitecustomize.py

It looks like that loading the instrumentation for celery segfaults when instrumentation mysql was loaded before (mysql currently does not load correctly see: open-telemetry/opentelemetry-python#858, so the issue will not occur until the fix is merged).
Modifying sitecustomize.py to:

entry_points = []
for entry_point in iter_entry_points("opentelemetry_instrumentor"):
    if entry_point.name not in ("mysql", "celery"):
        continue
    entry_points.append(entry_point)

def key(entry_point):
    return entry_point.name


entry_points = sorted(entry_points, key=key, reverse=True)

for entry_point in entry_points:
    try:
        entry_point.load()().instrument()  # type: ignore
        logger.debug("Instrumented %s", entry_point.name)

    except Exception:  # pylint: disable=broad-except
        logger.exception("Instrumenting of %s failed", entry_point.name)

makes the segfault reproducible every time (in py35 and py37, py38 works fine). Reversing the order in which the instrumentations are loaded does not raise a segfault.

Running:

gdb python
(gdb) run opentelemetry-instrumentation/src/opentelemetry/instrumentation/sitecustomize.py
...
(gdb) backtrace

says the following:

#0  0x000000000005c626 in ?? ()
open-telemetry/opentelemetry-python#1  0x00007ffff62013d8 in (anonymous namespace)::__future_category_instance() ()
   from /home/mario/workspaces/github/opentelemetry/opentelemetry-python/.venv/lib/python3.7/site-packages/_mysql_connector.cpython-37m-x86_64-linux-gnu.so
open-telemetry/opentelemetry-python#2  0x00007ffff6201769 in std::future_category() ()
   from /home/mario/workspaces/github/opentelemetry/opentelemetry-python/.venv/lib/python3.7/site-packages/_mysql_connector.cpython-37m-x86_64-linux-gnu.so
open-telemetry/opentelemetry-python#3  0x00007ffff5c0922d in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
open-telemetry/opentelemetry-python#4  0x00007ffff7fe102a in ?? () from /lib64/ld-linux-x86-64.so.2
open-telemetry/opentelemetry-python#5  0x00007ffff7fe1131 in ?? () from /lib64/ld-linux-x86-64.so.2
open-telemetry/opentelemetry-python#6  0x00007ffff7fe541e in ?? () from /lib64/ld-linux-x86-64.so.2
open-telemetry/opentelemetry-python#7  0x00007ffff7f1ed39 in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:196
open-telemetry/opentelemetry-python#8  0x00007ffff7fe49ba in ?? () from /lib64/ld-linux-x86-64.so.2
open-telemetry/opentelemetry-python#9  0x00007ffff7d9334c in dlopen_doit (a=a@entry=0x7fffffff3fb0) at dlopen.c:66
open-telemetry/opentelemetry-python#10 0x00007ffff7f1ed39 in __GI__dl_catch_exception (exception=exception@entry=0x7fffffff3f50, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:196
open-telemetry/opentelemetry-python#11 0x00007ffff7f1edd3 in __GI__dl_catch_error (objname=0xa04930, errstring=0xa04938, mallocedp=0xa04928, operate=<optimized out>, args=<optimized out>) at dl-error-skeleton.c:215
open-telemetry/opentelemetry-python#12 0x00007ffff7d93b59 in _dlerror_run (operate=operate@entry=0x7ffff7d932f0 <dlopen_doit>, args=args@entry=0x7fffffff3fb0) at dlerror.c:170
open-telemetry/opentelemetry-python#13 0x00007ffff7d933da in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
open-telemetry/opentelemetry-python#14 0x0000000000639011 in _PyImport_FindSharedFuncptr ()
open-telemetry/opentelemetry-python#15 0x0000000000645fe7 in _PyImport_LoadDynamicModuleWithSpec ()
open-telemetry/opentelemetry-python#16 0x0000000000646e80 in ?? ()
open-telemetry/opentelemetry-python#17 0x00000000005c8e2d in PyCFunction_Call ()
open-telemetry/opentelemetry-python#18 0x000000000053d13a in _PyEval_EvalFrameDefault ()
...

[github-actions]

This issue was marked stale due to lack of activity. It will be closed in 30 days.

[github-actions]

This issue was marked stale due to lack of activity. It will be closed in 30 days.

[github-actions]

Closed as inactive. Feel free to reopen if this issue needs resolving.