WebRTC related crash in domain server

[namark]

Stumbled upon this while testing out the WebRTC update for Linux #1406

(openssl_adapter.cc:786): SSL3 alert write:warning:close notify                                                                                                              [0/62485]
(dtls_transport.cc:682): DtlsTransport[0|1|_W]: DTLS transport closed by remote            
(jsep_transport_controller.cc:1344):  Transport 0 writability changed to 0.                
(peer_connection.cc:4698): Changing IceConnectionState 2 => 5                                                                                                                         
(srtp_transport.cc:366): The params in SRTP transport are reset.                                                                                                                      
(peer_connection.cc:4794): Session: 6208442725842442255 Old state: kStable New state: kClosed                                                                              
(peer_connection.cc:6741): Tearing down data channel transport for mid=0                                                                                                              
(sctp_transport.cc:305): UninitializeUsrSctp                                                                                                                                          
[Thread 0x7fffc77fe640 (LWP 2629225) exited]                                                                                                                                          
[Thread 0x7fffc7fff640 (LWP 2629224) exited]                                                                                                                                          
(openssl_stream_adapter.cc:912): Cleanup                                                                                                                                              
(peer_connection.cc:7011): Usage signature is 151526                         
(peer_connection.cc:1077): Session: 6208442725842442255 is destroyed.                                                                                                                 
malloc(): unsorted double linked list corrupted                                            
                                             
Thread 1 "domain-server" received signal SIGABRT, Aborted.                                 
0x00007ffff4a2fd22 in raise () from /usr/lib/libc.so.6                                     
(gdb) bt                                                                                                                                                                              
#0  0x00007ffff4a2fd22 in raise () from /usr/lib/libc.so.6                                                                                                                            
#1  0x00007ffff4a19862 in abort () from /usr/lib/libc.so.6                                                                                                                            
#2  0x00007ffff4a71d28 in __libc_message () from /usr/lib/libc.so.6                                                                                                                   
#3  0x00007ffff4a7992a in malloc_printerr () from /usr/lib/libc.so.6                                                                                                                  
#4  0x00007ffff4a7cf4c in _int_malloc () from /usr/lib/libc.so.6                                                                                                                      
#5  0x00007ffff4a7e4e1 in malloc () from /usr/lib/libc.so.6                                                                                                                           
#6  0x00007ffff795d463 in QArrayData::allocate(unsigned long, unsigned long, unsigned long, QFlags<QArrayData::AllocationOption>) () from /usr/lib/libQt5Core.so.5                    
#7  0x00007ffff79a95b5 in QByteArray::QByteArray(int, Qt::Initialization) () from /usr/lib/libQt5Core.so.5                                                                            
#8  0x00007ffff7993aa8 in QRingBuffer::reserve(long long) () from /usr/lib/libQt5Core.so.5                                                                                            
#9  0x00007ffff7a6e47a in QIODevicePrivate::read(char*, long long, bool) () from /usr/lib/libQt5Core.so.5                                                                             
#10 0x00007ffff7a6f046 in QIODevice::readAll() () from /usr/lib/libQt5Core.so.5         
#11 0x00007ffff660727a in ReceivedMessage::ReceivedMessage (this=0x5555558f24f0, packet=...) at /home/namark/dev/vircadia/libraries/networking/src/ReceivedMessage.cpp:38
#12 0x00007ffff65fefd6 in QSharedPointer<ReceivedMessage>::create<NLPacket&> () at /usr/include/qt/QtCore/qsharedpointer_impl.h:438
#13 0x00007ffff65fb587 in PacketReceiver::handleVerifiedPacket (this=0x5555558e8b70, packet=std::unique_ptr<udt::Packet> = {...})                          
    at /home/namark/dev/vircadia/libraries/networking/src/PacketReceiver.cpp:146                                                                                                      
#14 0x00007ffff6591f83 in operator() (__closure=0x55555591d4c8, packet=std::unique_ptr<udt::Packet> = {...})                                                                          
    at /home/namark/dev/vircadia/libraries/networking/src/LimitedNodeList.cpp:104          
#15 0x00007ffff659eede in std::__invoke_impl<void, LimitedNodeList::LimitedNodeList(int, int)::<lambda(std::unique_ptr<udt::Packet>)>&, std::unique_ptr<udt::Packet, std::default_delete<udt::Packet> > >(std::__invoke_other, struct {...} &) (__f=...) at /usr/include/c++/11.1.0/bits/invoke.h:61                                                                        
#16 0x00007ffff659e27a in std::__invoke_r<void, LimitedNodeList::LimitedNodeList(int, int)::<lambda(std::unique_ptr<udt::Packet>)>&, std::unique_ptr<udt::Packet, std::default_delete<udt::Packet> > >(struct {...} &) (__fn=...) at /usr/include/c++/11.1.0/bits/invoke.h:154
#17 0x00007ffff659da95 in std::_Function_handler<void(std::unique_ptr<udt::Packet, std::default_delete<udt::Packet> >), LimitedNodeList::LimitedNodeList(int, int)::<lambda(std::unique_ptr<udt::Packet, std::default_delete<udt::Packet> >)> >::_M_invoke(const std::_Any_data &, std::unique_ptr<udt::Packet, std::default_delete<udt::Packet> > &&) (__functor=...,      
    __args#0=...) at /usr/include/c++/11.1.0/bits/std_function.h:291
#18 0x00007ffff665be9f in std::function<void (std::unique_ptr<udt::Packet, std::default_delete<udt::Packet> >)>::operator()(std::unique_ptr<udt::Packet, std::default_delete<udt::Packet> >) const (this=0x55555591d4c8, __args#0=std::unique_ptr<udt::Packet> = {...}) at /usr/include/c++/11.1.0/bits/std_function.h:560
#19 0x00007ffff66598c1 in udt::Socket::readPendingDatagrams (this=0x55555591d3f0) at /home/namark/dev/vircadia/libraries/networking/src/udt/Socket.cpp:489                            
#20 0x00007ffff6660788 in QtPrivate::FunctorCall<QtPrivate::IndexesList<>, QtPrivate::List<>, void, void (udt::Socket::*)()>::call(void (udt::Socket::*)(), udt::Socket*, void**) (   
    f=(void (udt::Socket::*)(udt::Socket * const)) 0x7ffff6659250 <udt::Socket::readPendingDatagrams()>, o=0x55555591d3f0, arg=0x7fffffffd3a0)                                        
    at /usr/include/qt/QtCore/qobjectdefs_impl.h:152                                       
#21 0x00007ffff665f4c5 in QtPrivate::FunctionPointer<void (udt::Socket::*)()>::call<QtPrivate::List<>, void>(void (udt::Socket::*)(), udt::Socket*, void**) (                         
    f=(void (udt::Socket::*)(udt::Socket * const)) 0x7ffff6659250 <udt::Socket::readPendingDatagrams()>, o=0x55555591d3f0, arg=0x7fffffffd3a0)                                        
    at /usr/include/qt/QtCore/qobjectdefs_impl.h:185                 
#22 0x00007ffff665e121 in QtPrivate::QSlotObject<void (udt::Socket::*)(), QtPrivate::List<>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) (which=1,         
    this_=0x55555591c870, r=0x55555591d3f0, a=0x7fffffffd3a0, ret=0x0) at /usr/include/qt/QtCore/qobjectdefs_impl.h:418                                                               
#23 0x00007ffff7b7c8c5 in ?? () from /usr/lib/libQt5Core.so.5
#24 0x00007ffff64efe4b in NetworkSocket::readyRead (this=0x55555591d400)
    at /home/namark/dev/vircadia/build/libraries/networking/networking_autogen/CX623MDDSI/moc_NetworkSocket.cpp:228                                                                   
#25 0x00007ffff6647b0a in QtPrivate::FunctorCall<QtPrivate::IndexesList<>, QtPrivate::List<>, void, void (NetworkSocket::*)()>::call(void (NetworkSocket::*)(), NetworkSocket*, void**) (f=(void (NetworkSocket::*)(NetworkSocket * const)) 0x7ffff64efe22 <NetworkSocket::readyRead()>, o=0x55555591d400, arg=0x7fffffffd550)                                              
    at /usr/include/qt/QtCore/qobjectdefs_impl.h:152               
#26 0x00007ffff6647a48 in QtPrivate::FunctionPointer<void (NetworkSocket::*)()>::call<QtPrivate::List<>, void>(void (NetworkSocket::*)(), NetworkSocket*, void**) (                   
    f=(void (NetworkSocket::*)(NetworkSocket * const)) 0x7ffff64efe22 <NetworkSocket::readyRead()>, o=0x55555591d400, arg=0x7fffffffd550)                                             
    at /usr/include/qt/QtCore/qobjectdefs_impl.h:185       
#27 0x00007ffff66478c5 in QtPrivate::QSlotObject<void (NetworkSocket::*)(), QtPrivate::List<>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) (which=1, 
    this_=0x555555910a40, r=0x55555591d400, a=0x7fffffffd550, ret=0x0) at /usr/include/qt/QtCore/qobjectdefs_impl.h:418
#28 0x00007ffff7b7c8c5 in ?? () from /usr/lib/libQt5Core.so.5                             
#29 0x00007ffff7eda5f0 in ?? () from /usr/lib/libQt5Network.so.5                                                                                                                      
#30 0x00007ffff7eeddba in ?? () from /usr/lib/libQt5Network.so.5               
#31 0x00007ffff7b4519e in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /usr/lib/libQt5Core.so.5                                                          
#32 0x00007ffff7b9f736 in ?? () from /usr/lib/libQt5Core.so.5                                                                                                                         
#33 0x00007ffff41024dc in g_main_context_dispatch () from /usr/lib/libglib-2.0.so.0                                                                                                   
#34 0x00007ffff4156749 in ?? () from /usr/lib/libglib-2.0.so.0                  
#35 0x00007ffff40ffbc1 in g_main_context_iteration () from /usr/lib/libglib-2.0.so.0                                                                                                  
#36 0x00007ffff7b9eada in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQt5Core.so.5                                                 
#37 0x00007ffff7b43a6b in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQt5Core.so.5                                                                    
#38 0x00007ffff7b4c258 in QCoreApplication::exec() () from /usr/lib/libQt5Core.so.5                                                                                                   
#39 0x000055555572691d in main (argc=1, argv=0x7fffffffdfc8) at /home/namark/dev/vircadia/domain-server/src/main.cpp:38                    

Steps to reproduce: With domain server, assignment clients (-n 6) and an http server in the root of Web SDK all running locally, open a browser tab at localhost:8080, navigate to example/interface.html, click connect and wait for the CONNECED message, then close the tab. The domain server segfaults, assignment clients survive.

OS: Manjaro Linux
Browser: Firefox 93.0

[daleglass]

It's tricky to tell where such issues come from, please try building with memory debugging enabled. Unfortunately memory issues tend to be something else screwing things up, and malloc bumping into the corruption a while later.

[ctrlaltdavid]

Does this also happen on Windows or is it Linux-specific?

[namark]

Rebuilt with debug symbols and address sanitizer and it doesn't reproduce anymore of course -_-

[stale]

Hello! Is this still an issue?

[namark]

Can't reproduce anymore, probably the same crash as described in vircadia/vircadia-web-sdk#61