connection stuck after internet loss and recovery

[leventov]

Origin: golang/go#41549

Stack trace:

1 @ 0x48608 0x412e8 0x75ffc 0xde6a8 0xdf670 0xdf655 0x1d1ab0 0x1e24e8 0x21a5ec 0x10f724 0x21a834 0x2180a4 0x21d074 0x21d07d 0xda238 0x4cd528 0x4cd4fd 0x4e54e8 0x7ad2c
--
  | #	0x75ffb		internal/poll.runtime_pollWait+0x43				runtime/netpoll.go:220 |   | #	0x75ffb		internal/poll.runtime_pollWait+0x43				runtime/netpoll.go:220
  | #	0xde6a7		internal/poll.(*pollDesc).wait+0x2f				internal/poll/fd_poll_runtime.go:87 |   | #	0xde6a7		internal/poll.(*pollDesc).wait+0x2f				internal/poll/fd_poll_runtime.go:87
  | #	0xdf66f		internal/poll.(*pollDesc).waitRead+0x17b			internal/poll/fd_poll_runtime.go:92 |   | #	0xdf66f		internal/poll.(*pollDesc).waitRead+0x17b			internal/poll/fd_poll_runtime.go:92
  | #	0xdf654		internal/poll.(*FD).Read+0x160					internal/poll/fd_unix.go:159 |   | #	0xdf654		internal/poll.(*FD).Read+0x160					internal/poll/fd_unix.go:159
  | #	0x1d1aaf	net.(*netFD).Read+0x37						net/fd_posix.go:55 |   | #	0x1d1aaf	net.(*netFD).Read+0x37						net/fd_posix.go:55
  | #	0x1e24e7	net.(*conn).Read+0x63						net/net.go:182 |   | #	0x1e24e7	net.(*conn).Read+0x63						net/net.go:182
  | #	0x21a5eb	crypto/tls.(*atLeastReader).Read+0x77				crypto/tls/conn.go:779 |   | #	0x21a5eb	crypto/tls.(*atLeastReader).Read+0x77				crypto/tls/conn.go:779
  | #	0x10f723	bytes.(*Buffer).ReadFrom+0xa3					bytes/buffer.go:204 |   | #	0x10f723	bytes.(*Buffer).ReadFrom+0xa3					bytes/buffer.go:204
  | #	0x21a833	crypto/tls.(*Conn).readFromUntil+0xc3				crypto/tls/conn.go:801 |   | #	0x21a833	crypto/tls.(*Conn).readFromUntil+0xc3				crypto/tls/conn.go:801
  | #	0x2180a3	crypto/tls.(*Conn).readRecordOrCCS+0xfb				crypto/tls/conn.go:608 |   | #	0x2180a3	crypto/tls.(*Conn).readRecordOrCCS+0xfb				crypto/tls/conn.go:608
  | #	0x21d073	crypto/tls.(*Conn).readRecord+0x14f				crypto/tls/conn.go:576 |   | #	0x21d073	crypto/tls.(*Conn).readRecord+0x14f				crypto/tls/conn.go:576
  | #	0x21d07c	crypto/tls.(*Conn).Read+0x158					crypto/tls/conn.go:1252 |   | #	0x21d07c	crypto/tls.(*Conn).Read+0x158					crypto/tls/conn.go:1252
  | #	0xda237		io.ReadAtLeast+0x6b						io/io.go:314 |   | #	0xda237		io.ReadAtLeast+0x6b						io/io.go:314
  | #	0x4cd527	io.ReadFull+0x67						io/io.go:333 |   | #	0x4cd527	io.ReadFull+0x67						io/io.go:333
  | #	0x4cd4fc	github.com/eclipse/paho.mqtt.golang/packets.ReadPacket+0x3c	github.com/eclipse/paho.mqtt.golang@v1.2.0/packets/packets.go:105 |   | #	0x4cd4fc	github.com/eclipse/paho.mqtt.golang/packets.ReadPacket+0x3c	github.com/eclipse/paho.mqtt.golang@v1.2.0/packets/packets.go:105
  | #	0x4e54e7	github.com/eclipse/paho%2emqtt%2egolang.incoming+0xe7		github.com/eclipse/paho.mqtt.golang@v1.2.0/net.go:132 |   | #	0x4e54e7	github.com/eclipse/paho%2emqtt%2egolang.incoming+0xe7		github.com/eclipse/paho.mqtt.golang@v1.2.0/net.go:132

Should probably add a readTimeout option to workaround this.

[MattBrittan]

Hi @leventov, It looks like you are using v1.2.0; that is the most recent release but significant changes have been made since its release (over a year ago) and I believe you will find @master to be more stable (go get github.com/eclipse/paho.mqtt.golang@master) . Dealing with disconnections that the OS does not report is difficult; enabling keepalives is probably the best option (this will stop the read by closing the connection if no response is received).

[magiconair]

@MattBrittan is there a reason there is no new release? Tracking master is a bit rough.

[MattBrittan]

@magiconair Sorry - I'm not a committer (yet - paperwork is in progress). I believe that @master is currently stable so a release is warranted and hopefully it will not be too long.

[jwzl]

I have same problem

[MattBrittan]

@jwzl - have you followed the advice above (as there has been no response I can only assume the original issue is fixed). Unless your issue is identical to the above (and not fixed by upgrading and enabling keepalives) please log a new issue with full details if you would like help (including logs etc).

[jwzl]

hello, MattBrittan,
I have go get github.com/eclipse/paho.mqtt.golang@master and get the newest commit ;
and keepalives = 12;

but when I unplug my ethernet cable, the ConnectionLostHandler is not called.
I plan to do re-connect the broker in onlost handle, but it never reach.

[jwzl]

my linux OS is X86 64 ubuntu16.04

[MattBrittan]

@jwzl as I said please log a new issue. There are a large number of things that could cause an issue like this (some of them in your code) and as you are using the latest version (with major changes since v1.2.0;) I doubt your issue is the same as this one. In your new issue please:

• Provide logs obtained when the issue is occurring (see my response on this issue for instructions on enabling logging). This does not always provide enough information but without logs or an application that reproduces the fault there is little anyone can do to help you.
• Include a Minimal, Reproducible Example if possible. It may not be possible for me to replicate the issue on my PC but at least I will have a full understanding of what options you are using.

[jwzl]

Thank， It's okay now!

[MattBrittan]

Closing this it appears the original issue has been resolved in @master (will issue a new release shortly).