Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding Geom to FIXMEModules #9

Open
wants to merge 11 commits into
base: cxxmodules_global_index
Choose a base branch
from
Open

Adding Geom to FIXMEModules #9

wants to merge 11 commits into from

Conversation

arpi-r
Copy link

@arpi-r arpi-r commented Jul 30, 2019

Fixes a failing test

vgvassilev and others added 11 commits July 25, 2019 22:40
@vgvassilev
[cxxmodules] Implement global module indexing to improve performance.
d52f540 
The global module index represents an efficient on-disk hash table which stores
identifier->module mapping. Every time clang finds a unknown identifier we
are informed and we can load the corresponding module on demand.

This way we can provide minimal set of loaded modules. Currently, we see that
for hsimple.C only the half of the modules are loaded. This can be further
improved because we currently load all modules which have an identifier, that is
when looking for (for example TPad) we will load all modules which have the
identifier TPad, including modules which contain only a forward declaration of
it.

Kudos Arpitha Raghunandan (arpi-r)!
@vgvassilev
[cxxmodules] Do not cache the file lookup failure.
40c2d32 
In cases where we have the module cache path and prebuilt module path pointing
to the same location, the FileManager should not cache the module file lookup
failure because it may currently be building the module.

This patch is necessary because the global module index is built from the
module cache path and it is loaded from the prebuilt module path.

In a full explicit or implicit module build infrastructure this is not a problem.
However, in a mixed scenario (where modules for third-party dependencies are
built implicitly) such as ours this is problematic. One of the reasons is that
we cannot configure the prebuilt modules paths or module cache paths externally.
This is because the interpreter (at construction time) #includes RuntimeUniverse
which may trigger module build.

This patch allows us to refactor some of the code working around this issue.
@vgvassilev
[cxxmodules] Preload only common modules.
fe902d6
@vgvassilev
[cxxmodules][cling] Add a callback for start/finish code generation.
436e2c2 
When we are generating code, CodeGen automatically tries to complete decl's
redeclaration chain. This ends up a call to the external sources, one of which
is our global module index (GMI).

The semantics of the GMI is to aid the frontend, that is to automatically
import missing modules based on the typed *by the user* identifiers. It does
not intend to aid the code generation by any means. Currently this happens
when completing chains for existing identifiers such as 'volume' which happens
to be an indentifier part of TMVA.pcm, too. Thus, CodeGen unintentionally loads
the module of TMVA. This is an overkill, but worse -- it brings recursiveness
to the identifier resolution system.

This patch disables looking in the GMI at codegen time.
@vgvassilev
[tcling] Modernize header file virtual -> override.
e0c00f7
@vgvassilev
[tcling] Modernize header file: use inline initialization.
eface7c
@vgvassilev
[cxxmodules] Tighten the findInGlobalModuleIndex routine.
22b9c5f 
All reasibility checks are done in the routine itself and we also call it,
when we need to complete type through LookupObject(TagDecl*).
@vgvassilev
[cxxmodules] Preload Hist because it has the same issue as Gpad.
2e08912 
The issue is that, it does not contain the identifier for TH1.
@arpi-r @vgvassilev
Preload Graf for the same reason as Hist and Gpad
c5db1a4 
It does not contain identifier TFrame
@arpi-r @vgvassilev
Adding GenVector and ROOTDataFrame to FIXMEModules
5e11567 
Fixes some failing tests
@arpi-r
Adding Geom to FIXMEModules
c410f9d 
Fixes a failing test
@vgvassilev vgvassilev force-pushed the cxxmodules_global_index branch 3 times, most recently from 5c51e52 to 31efb93 Compare October 25, 2019 12:55
@vgvassilev vgvassilev force-pushed the cxxmodules_global_index branch 2 times, most recently from 9bcf30e to 48baaf7 Compare December 2, 2019 08:07
@vgvassilev vgvassilev force-pushed the cxxmodules_global_index branch 2 times, most recently from ab3d66f to f554c86 Compare January 22, 2020 12:22
@vgvassilev vgvassilev force-pushed the cxxmodules_global_index branch 6 times, most recently from 8f0cbc5 to 30d0eff Compare January 29, 2020 06:35
@vgvassilev vgvassilev force-pushed the cxxmodules_global_index branch 2 times, most recently from 7d4aee1 to 774c08f Compare February 24, 2020 08:51
vgvassilev pushed a commit that referenced this pull request May 7, 2021
@Axel-Naumann
[cling] Lifetime of MetaParser is that of its input:
3aa83e5 
Before, MetaParser might have pointed to a StringRef whose storage
was gone, see asan failure in roottest/cling/other/runfileClose.C below.

This was caused by recursive uses of MetaParser; see stack trace below:
the inner recursion returned, but as the same MetaParser object was used
by both frames, the objects cursor now pointed to freed memory.

Instead, create a MetaParser (and MetaLexer) object per input. That way,
their lifetime corresponds to the lifetime of their input.

=================================================================
==529104==ERROR: AddressSanitizer: stack-use-after-return on address 0x7ffff3afd82a at pc 0x7fffea18df6d bp 0x7fffffff8170 sp 0x7fffffff8168
READ of size 1 at 0x7ffff3afd82a thread T0
[Detaching after fork from child process 529183]
    #0 0x7fffea18df6c in cling::MetaLexer::Lex(cling::Token&) src/interpreter/cling/lib/MetaProcessor/MetaLexer.cpp:58:11
    #1 0x7fffea190d7c in cling::MetaParser::lookAhead(unsigned int) src/interpreter/cling/lib/MetaProcessor/MetaParser.cpp:89:15
    #2 0x7fffea190bd5 in cling::MetaParser::consumeToken() src/interpreter/cling/lib/MetaProcessor/MetaParser.cpp:49:5
    #3 0x7fffea191d4d in cling::MetaParser::isLCommand(cling::MetaSema::ActionResult&) src/interpreter/cling/lib/MetaProcessor/MetaParser.cpp:147:9
    #4 0x7fffea1914dd in cling::MetaParser::isCommand(cling::MetaSema::ActionResult&, cling::Value*) src/interpreter/cling/lib/MetaProcessor/MetaParser.cpp:123:12
    #5 0x7fffea191216 in cling::MetaParser::isMetaCommand(cling::MetaSema::ActionResult&, cling::Value*) src/interpreter/cling/lib/MetaProcessor/MetaParser.cpp:101:33
    #6 0x7fffea14e5aa in cling::MetaProcessor::process(llvm::StringRef, cling::Interpreter::CompilationResult&, cling::Value*, bool) src/interpreter/cling/lib/MetaProcessor/MetaProcessor.cpp:317:24
    #7 0x7fffe99b67b7 in HandleInterpreterException(cling::MetaProcessor*, char const*, cling::Interpreter::CompilationResult&, cling::Value*) src/core/metacling/src/TCling.cxx:2431:29
    #8 0x7fffe99bde30 in TCling::Load(char const*, bool) src/core/metacling/src/TCling.cxx:3454:10
    #9 0x7ffff7865f11 in TSystem::Load(char const*, char const*, bool) src/core/base/src/TSystem.cxx:1941:27
    #10 0x7ffff7b8a0e3 in TUnixSystem::Load(char const*, char const*, bool) src/core/unix/src/TUnixSystem.cxx:2789:20
    #11 0x7fffd78dd08b  (<unknown module>)
    #12 0x7fffe9f8a5d9 in cling::IncrementalExecutor::executeWrapper(llvm::StringRef, cling::Value*) const src/interpreter/cling/lib/Interpreter/IncrementalExecutor.cpp:376:3
    root-project#13 0x7fffe9d73dc2 in cling::Interpreter::RunFunction(clang::FunctionDecl const*, cling::Value*) src/interpreter/cling/lib/Interpreter/Interpreter.cpp:1141:20
    root-project#14 0x7fffe9d6e317 in cling::Interpreter::EvaluateInternal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cling::CompilationOptions, cling::Value*, cling::Transaction**, unsigned long) src/interpreter/cling/lib/Interpreter/Interpreter.cpp:1391:29
    root-project#15 0x7fffe9d6c1fe in cling::Interpreter::process(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cling::Value*, cling::Transaction**, bool) src/interpreter/cling/lib/Interpreter/Interpreter.cpp:819:9
    root-project#16 0x7fffea151826 in cling::MetaProcessor::readInputFromFile(llvm::StringRef, cling::Value*, unsigned long, bool) src/interpreter/cling/lib/MetaProcessor/MetaProcessor.cpp:507:22
    root-project#17 0x7fffe99b585b in TCling::ProcessLine(char const*, TInterpreter::EErrorCode*) src/core/metacling/src/TCling.cxx:2570:39
    root-project#18 0x7fffe99bbfee in TCling::ProcessLineSynch(char const*, TInterpreter::EErrorCode*) src/core/metacling/src/TCling.cxx:3496:17
    root-project#19 0x7ffff77203d3 in TApplication::ExecuteFile(char const*, int*, bool) src/core/base/src/TApplication.cxx:1608:30
    root-project#20 0x7ffff771ebdf in TApplication::ProcessFile(char const*, int*, bool) src/core/base/src/TApplication.cxx:1480:11
    root-project#21 0x7ffff771e385 in TApplication::ProcessLine(char const*, bool, int*) src/core/base/src/TApplication.cxx:1453:14
    root-project#22 0x7ffff7f8157a in TRint::ProcessLineNr(char const*, char const*, int*) src/core/rint/src/TRint.cxx:766:11
    root-project#23 0x7ffff7f802f0 in TRint::Run(bool) src/core/rint/src/TRint.cxx:424:22
    root-project#24 0x4ff96d in main src/main/src/rmain.cxx:30:12
    root-project#25 0x7ffff6e040b2 in __libc_start_main /build/glibc-YbNSs7/glibc-2.31/csu/../csu/libc-start.c:308:16
    root-project#26 0x41f35d in _start (asan/bin/root.exe+0x41f35d)

Address 0x7ffff3afd82a is located in stack of thread T0 at offset 42 in frame
    #0 0x7fffe99b3d8f in TCling::ProcessLine(char const*, TInterpreter::EErrorCode*) src/core/metacling/src/TCling.cxx:2456

  This frame has 21 object(s):
    [32, 56) 'sLine' (line 2462) <== Memory access at offset 42 is inside this variable
    [96, 104) 'R__guard2471' (line 2471)
    [128, 136) 'R__guard2488' (line 2488)
    [160, 176) 'interpreterFlagsRAII' (line 2491)
    [192, 240) 'result' (line 2511)
    [272, 276) 'compRes' (line 2512)
    [288, 312) 'mod_line' (line 2517)
    [352, 376) 'aclicMode' (line 2518)
    [416, 440) 'arguments' (line 2519)
    [480, 504) 'io' (line 2520)
    [544, 568) 'fname' (line 2521)
    [608, 632) 'ref.tmp' (line 2547)
    [672, 696) 'ref.tmp145' (line 2547)
    [736, 768) 'code' (line 2555)
    [800, 832) 'codeline' (line 2556)
    [864, 1384) 'in' (line 2559)
    [1520, 1552) 'ref.tmp176' (line 2562)
    [1584, 1600) 'agg.tmp'
    [1616, 1624) 'ref.tmp198' (line 2568)
    [1648, 1664) 'agg.tmp207'
    [1680, 1696) 'autoParseRaii' (line 2588)
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-use-after-return src/interpreter/cling/lib/MetaProcessor/MetaLexer.cpp:58:11 in cling::MetaLexer::Lex(cling::Token&)
Shadow bytes around the buggy address:
  0x10007e757ab0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
  0x10007e757ac0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
  0x10007e757ad0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
  0x10007e757ae0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
  0x10007e757af0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
=>0x10007e757b00: f5 f5 f5 f5 f5[f5]f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
  0x10007e757b10: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
  0x10007e757b20: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
  0x10007e757b30: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
  0x10007e757b40: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
  0x10007e757b50: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==529104==ABORTING

    at src/interpreter/cling/lib/MetaProcessor/MetaLexer.cpp:49
    at src/interpreter/cling/lib/MetaProcessor/MetaParser.cpp:41
    compRes=@0x7ffff3afd910: cling::Interpreter::kSuccess, result=0x7ffff3afd8c0, disableValuePrinting=false)
    at src/interpreter/cling/lib/MetaProcessor/MetaProcessor.cpp:314
    input_line=0x7ffff3afd829 "#define XYZ 21", compRes=@0x7ffff3afd910: cling::Interpreter::kSuccess,
    result=0x7ffff3afd8c0) at src/core/metacling/src/TCling.cxx:2431
    error=0x7fffd78cb0f4 <x>) at src/core/metacling/src/TCling.cxx:2591
    sync=false, err=0x7fffd78cb0f4 <x>) at src/core/base/src/TApplication.cxx:1472
    line=0x7fffd78c9000 "#define XYZ 21", error=0x7fffd78cb0f4 <x>)
    at src/core/base/src/TROOT.cxx:2328
   from asan/roottest/cling/other/fileClose_C.so
    filename=0x6070000f0fd0 "asan/roottest/cling/other/fileClose_C.so", flag=257)
    at /home/axel/build/llvm/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:6270
    at src/interpreter/cling/lib/Utils/PlatformPosix.cpp:118
    permanent=false, resolved=true)
    at src/interpreter/cling/lib/Interpreter/DynamicLibraryManager.cpp:184
    at src/interpreter/cling/lib/Interpreter/Interpreter.cpp:1444
    T=0x0) at src/interpreter/cling/lib/Interpreter/Interpreter.cpp:1560
    at src/interpreter/cling/lib/MetaProcessor/MetaSema.cpp:57
    actionResult=@0x7ffff39532b0: cling::MetaSema::AR_Success)
vgvassilev pushed a commit that referenced this pull request Nov 5, 2021
@eguiraud
[DF] Avoid crash in tutorial due to reused canvas name
89a658e 
This tutorial crashed with the following stacktrace when run
interactively. Avoiding to register a canvas with the same name
multiple times fixes the crash:

```
    #8  0x00007f5b7876967d in TCanvas::Resize(char const*) (this=0x55e768e126c0) at ../graf2d/gpad/src/TCanvas.cxx:1740
    #9  0x00007f5b3e90d668 in TRootCanvas::HandleContainerConfigure(Event_t*) (this=0x55e76852b460) at ../gui/gui/src/TRootCanvas.cxx:1789
    #10 0x00007f5b3e8464fd in TGFrame::HandleEvent(Event_t*) (this=0x55e767938e70, event=0x7f5b79adff40) at ../gui/gui/src/TGFrame.cxx:476
    #11 0x00007f5b3e7f4c9a in TGClient::HandleEvent(Event_t*) (this=0x55e768de2290, event=0x7f5b79adff40) at ../gui/gui/src/TGClient.cxx:846
    #12 0x00007f5b3e7f531d in TGClient::ProcessOneEvent() (this=0x55e768de2290) at ../gui/gui/src/TGClient.cxx:656
    root-project#13 TGClient::ProcessOneEvent() (this=0x55e768de2290) at ../gui/gui/src/TGClient.cxx:648
    root-project#14 0x00007f5b3e7f536b in TGClient::HandleInput() (this=0x55e768de2290) at ../gui/gui/src/TGClient.cxx:703
    root-project#15 0x00007f5b8dcb0ff8 in TUnixSystem::DispatchOneEvent(bool) (this=0x55e75ccfd080, pendingOnly=<optimized out>) at ../core/unix/src/TUnixSystem.cxx:1067
    root-project#16 0x00007f5b8dbd0dca in TSystem::ProcessEvents() (this=0x55e75ccfd080) at ../core/base/src/TSystem.cxx:424
    root-project#17 0x00007f5b8130600d in  ()
    root-project#18 0x00007f5b79ae0450 in  ()
    root-project#19 0x00007f5b8de5215f in WrapperCall(Cppyy::TCppMethod_t, size_t, void*, void*, void*) (method=94452242807424, nargs=0, args_=0x7f5b79ae01d7, self=0x55e75ccfd080, result=0x7f5b79ae01d7) at ../bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:778
    root-project#20 0x00007f5b8de527cf in CallT<unsigned char> (args=<optimized out>, nargs=<optimized out>, self=<optimized out>, method=<optimized out>) at ../bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:816
    root-project#21 Cppyy::CallB(long, void*, unsigned long, void*) (method=<optimized out>, self=<optimized out>, nargs=<optimized out>, args=<optimized out>) at ../bindings/pyroot/cppyy/cppyy-backend/clingwrapper/src/clingwrapper.cxx:833
    root-project#22 0x00007f5b8decdc0f in GILCallB (ctxt=0x7f5b79ae0430, self=<optimized out>, method=<optimized out>) at ../bindings/pyroot/cppyy/CPyCppyy/src/Executors.cxx:69
    root-project#23 CPyCppyy::(anonymous namespace)::BoolExecutor::Execute(Cppyy::TCppMethod_t, Cppyy::TCppObject_t, CPyCppyy::CallContext*) (this=<optimized out>, method=<optimized out>, self=<optimized out>, ctxt=0x7f5b79ae0430) at ../bindings/pyroot/cppyy/CPyCppyy/src/Executors.cxx:148
    root-project#24 0x00007f5b8deba4c9 in CPyCppyy::CPPMethod::ExecuteFast(void*, long, CPyCppyy::CallContext*) (self=<optimized out>, offset=<optimized out>, ctxt=<optimized out>, this=<optimized out>, this=<optimized out>) at ../bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:74
    root-project#25 0x00007f5b8debd3a8 in CPyCppyy::CPPMethod::ExecuteProtected(void*, long, CPyCppyy::CallContext*) (this=this entry=0x55e760617f50, self=0x55e75ccfd080, offset=0, ctxt=0x7f5b79ae0430) at ../bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:149
    root-project#26 0x00007f5b8debb6fa in CPyCppyy::CPPMethod::Execute(void*, long, CPyCppyy::CallContext*) (this=this entry=0x55e760617f50, self=self entry=0x55e75ccfd080, offset=<optimized out>, ctxt=ctxt entry=0x7f5b79ae0430) at ../bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:728
    root-project#27 0x00007f5b8debc46c in CPyCppyy::CPPMethod::Call(CPyCppyy::CPPInstance*&, _object*, _object*, CPyCppyy::CallContext*) (this=0x55e760617f50, self= 0x7f5b8080ef50: 0x7f5b808043c0, args=0x7f5b8e1ab040, kwds=<optimized out>, ctxt=0x7f5b79ae0430) at ../bindings/pyroot/cppyy/CPyCppyy/src/CPPMethod.cxx:783
    root-project#28 0x00007f5b8dec09fe in CPyCppyy::(anonymous namespace)::mp_call(CPyCppyy::CPPOverload*, PyObject*, PyObject*) (pymeth=0x7f5b8080ef40, args=0x7f5b8e1ab040, kwds=0x0) at ../bindings/pyroot/cppyy/CPyCppyy/src/CPPOverload.cxx:566
    root-project#29 0x00007f5b8e941333 in _PyObject_MakeTpCall () at /usr/lib/libpython3.9.so.1.0
    root-project#30 0x00007f5b8e93d218 in _PyEval_EvalFrameDefault () at /usr/lib/libpython3.9.so.1.0
    root-project#31 0x00007f5b8e936fd9 in  () at /usr/lib/libpython3.9.so.1.0
    root-project#32 0x00007f5b8e948b8e in _PyFunction_Vectorcall () at /usr/lib/libpython3.9.so.1.0
    root-project#33 0x00007f5b8e93aec9 in _PyEval_EvalFrameDefault () at /usr/lib/libpython3.9.so.1.0
    root-project#34 0x00007f5b8e94896b in _PyFunction_Vectorcall () at /usr/lib/libpython3.9.so.1.0
    root-project#35 0x00007f5b8e93858e in _PyEval_EvalFrameDefault () at /usr/lib/libpython3.9.so.1.0
    root-project#36 0x00007f5b8e94896b in _PyFunction_Vectorcall () at /usr/lib/libpython3.9.so.1.0
    root-project#37 0x00007f5b8e93858e in _PyEval_EvalFrameDefault () at /usr/lib/libpython3.9.so.1.0
    root-project#38 0x00007f5b8e94896b in _PyFunction_Vectorcall () at /usr/lib/li