Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add task extraction & caching #27

Merged
merged 5 commits into from
Feb 14, 2022
Merged

Add task extraction & caching #27

merged 5 commits into from
Feb 14, 2022

Conversation

junrushao
Copy link
Owner

No description provided.

@junrushao
Copy link
Owner Author

ResNeXt @ CUDA failed

[53 / 114] Extract resnext_50, input_shape = [8, 3, 240, 240]                                                                                                                                                                                                                   Load from cache: /root/dataset/relay-models/resnext_50-8,3,240,240.json
Exception in thread Thread-167:                                                                                                                                                                                                                                                 Traceback (most recent call last):
  File "/root/miniconda/envs/python38/lib/python3.8/threading.py", line 932, in _bootstrap_inner                                                                                                                                                                                    self.run()
  File "/root/miniconda/envs/python38/lib/python3.8/threading.py", line 870, in run                                                                                                                                                                                                 self._target(*self._args, **self._kwargs)
  File "/root/Projects/tvm-dev/python/tvm/meta_schedule/integration.py", line 254, in _func                                                                                                                                                                                         compiler.lower(mod, target)
  File "/root/Projects/tvm-dev/python/tvm/relay/backend/vm.py", line 155, in lower                                                                                                                                                                                                  self._lower(mod, target, target_host)
  File "/root/Projects/tvm-dev/python/tvm/_ffi/_ctypes/packed_func.py", line 237, in __call__                                                                                                                                                                                       raise get_last_ffi_error()
tvm._ffi.base.TVMError: Traceback (most recent call last):                                                                                                                                                                                                                        78: TVMFuncCall                                                                                                                                                                                                                                                                       at /root/Projects/tvm-dev/src/runtime/c_runtime_api.cc:475                                                                                                                                                                                                                77: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1151
  76: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const                                                                                                                                      at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:622                                                                                                                                                                             75: std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::relay::vm::VMCompiler::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_0>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:291
  74: std::enable_if<__and_<std::is_void<void>, std::__is_invocable<tvm::relay::vm::VMCompiler::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_0&, tvm::runtime::T
VMArgs, tvm::runtime::TVMRetValue*> >::value, void>::type std::__invoke_r<void, tvm::relay::vm::VMCompiler::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_0&, tvm
::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(tvm::relay::vm::VMCompiler::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_0&, tvm::runtime::TVMArgs&&, tvm::runti
me::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:153
  73: void std::__invoke_impl<void, tvm::relay::vm::VMCompiler::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_0&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValu
e*>(std::__invoke_other, tvm::relay::vm::VMCompiler::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_0&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60
  72: tvm::relay::vm::VMCompiler::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_0::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /root/Projects/tvm-dev/src/relay/backend/vm/compiler.cc:833
  71: tvm::relay::vm::VMCompiler::Lower(tvm::IRModule, tvm::runtime::Map<tvm::Integer, tvm::Target, void, void>, tvm::Target)
        at /root/Projects/tvm-dev/src/relay/backend/vm/compiler.cc:884
  70: tvm::relay::vm::VMCompiler::OptimizeModuleImpl(tvm::IRModule)
        at /root/Projects/tvm-dev/src/relay/backend/vm/compiler.cc:1112
  69: tvm::transform::Pass::operator()(tvm::IRModule) const
        at /root/Projects/tvm-dev/src/ir/transform.cc:255
  68: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
        at /root/Projects/tvm-dev/src/ir/transform.cc:267
  67: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
        at /root/Projects/tvm-dev/src/ir/transform.cc:487
  66: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
        at /root/Projects/tvm-dev/src/ir/transform.cc:267
  65: tvm::relay::transform::FunctionPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const
        at /root/Projects/tvm-dev/src/relay/ir/transform.cc:133
  64: tvm::runtime::TypedPackedFunc<tvm::relay::Function (tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext) const
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1498
  63: tvm::relay::Function tvm::runtime::detail::typed_packed_call_dispatcher<tvm::relay::Function>::run<tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::relay::Function&&, tvm::IRModule&&, tvm::transform::PassContext
&&)
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1444
  62: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext>(tvm::relay::Function&&, tvm::IRModule&&, tvm::transform::PassContext&&) const
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1369
  61: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:622
  60: std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::runtime::TypedPackedFunc<tvm::relay::Function (tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::AlterOpLayout()::$_1>(tv
m::relay::transform::AlterOpLayout()::$_1)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)                                                                              at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:291
  59: std::enable_if<__and_<std::is_void<void>, std::__is_invocable<tvm::runtime::TypedPackedFunc<tvm::relay::Function (tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::AlterOpLayout()::$_1>(tvm::relay::transform::AlterOpLayout()::$_1)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*> >::value, void>::type std::__invoke_r<void, tvm::runtime::TypedPackedFunc<tvm::relay::Function (tvm::relay::Function, tvm::IR
Module, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::AlterOpLayout()::$_1>(tvm::relay::transform::AlterOpLayout()::$_1)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(tvm::runtime::TypedPackedFunc<tvm::relay::Function (tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::AlterOpLayout()::$_1>(tvm::relay::transform::AlterOpLayout()::$_1)::{lambda(tvm::runtime::TVMArgs const&, tvm::ru
ntime::TVMRetValue*)#1}&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)                                                                                                                                                                                                        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:153
  58: void std::__invoke_impl<void, tvm::runtime::TypedPackedFunc<tvm::relay::Function (tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::AlterOpLayout()::$_1>(tvm::relay::transform::AlterOpLayout()::$_1)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(std::__invoke_other, tvm::runtime::TypedPackedFunc<tvm::relay::Function (tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambd
a<tvm::relay::transform::AlterOpLayout()::$_1>(tvm::relay::transform::AlterOpLayout()::$_1)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)                                                              at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60
  57: tvm::runtime::TypedPackedFunc<tvm::relay::Function (tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::AlterOpLayout()::$_1>(tvm::relay::transform::AlterOpLayout()::$_1)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*) const
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1492                                                                                                                                                                                                          56: void tvm::runtime::detail::unpack_call<tvm::relay::Function, 3, tvm::relay::transform::AlterOpLayout()::$_1>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const*, tvm::relay::transform::AlterOpLayout()::$_1 const&, tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)                                                                                                                                                                                                                                                at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1421
  55: void tvm::runtime::detail::unpack_call_dispatcher<tvm::relay::Function, 3, 0, tvm::relay::transform::AlterOpLayout()::$_1>::run<>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const*, tvm::relay::transform::AlterOpLayout()::$_1 cons
t&, tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)                                                                                                                                                                                                                           at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1382                                                                                                                                                                                                          54: void tvm::runtime::detail::unpack_call_dispatcher<tvm::relay::Function, 2, 1, tvm::relay::transform::AlterOpLayout()::$_1>::run<tvm::runtime::TVMMovableArgValueWithContext_>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const*, tvm::relay::transform::AlterOpLayout()::$_1 const&, tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*, tvm::runtime::TVMMovableArgValueWithContext_&&)
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1382
  53: void tvm::runtime::detail::unpack_call_dispatcher<tvm::relay::Function, 1, 2, tvm::relay::transform::AlterOpLayout()::$_1>::run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>(std::__cxx11::basic_string<char, std::char_tra
its<char>, std::allocator<char> > const*, tvm::relay::transform::AlterOpLayout()::$_1 const&, tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*, tvm::runtime::TVMMovableArgValueWithContext_&&, tvm::runtime::TVMMovableArgValueWithContext_&&)
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1382
  52: void tvm::runtime::detail::unpack_call_dispatcher<tvm::relay::Function, 0, 3, tvm::relay::transform::AlterOpLayout()::$_1>::run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>(
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const*, tvm::relay::transform::AlterOpLayout()::$_1 const&, tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*, tvm::runtime::TVMMovableArgValueWithContext_&&, tvm::runtime::TVMMovableAr
gValueWithContext_&&, tvm::runtime::TVMMovableArgValueWithContext_&&)
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1397
  51: tvm::relay::transform::AlterOpLayout()::$_1::operator()(tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext) const
        at /root/Projects/tvm-dev/src/relay/transforms/alter_op_layout.cc:134
  50: tvm::relay::alter_op_layout::AlterOpLayout(tvm::RelayExpr const&)
        at /root/Projects/tvm-dev/src/relay/transforms/alter_op_layout.cc:124
  49: tvm::relay::ForwardRewrite(tvm::RelayExpr const&, tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)> const&, std::function<tvm::runtime::ObjectRef (tvm::relay::Cal
l const&)>, std::function<tvm::RelayExpr (tvm::RelayExpr const&)>)
        at /root/Projects/tvm-dev/src/relay/transforms/forward_rewrite.cc:180
  48: tvm::relay::ForwardRewriter::Rewrite(tvm::RelayExpr const&)
        at /root/Projects/tvm-dev/src/relay/transforms/forward_rewrite.cc:71
  47: tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)
        at /root/Projects/tvm-dev/src/relay/ir/expr_functor.cc:102
  46: void tvm::relay::ExpandDataflow<tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3>(tvm::RelayExpr, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay:
:MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3)
        at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:492
  45: void tvm::relay::ExpandDataflow<tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3, tvm::relay::ExpandDataflow<tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2,
tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3>(tvm::RelayExpr, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3)::{lambda(tvm::RelayExpr const&)#1}>(tvm::RelayExp
r, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3, tvm::relay::ExpandDataflow<tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::Visit
Expr(tvm::RelayExpr const&)::$_3>(tvm::RelayExpr, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3)::{lambda(tvm::RelayExpr const&)#1})
        at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:462
  44: tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3::operator()(tvm::RelayExpr const&) const
        at /root/Projects/tvm-dev/src/relay/ir/expr_functor.cc:98
  43: tvm::relay::MixedModeMutator::VisitLeaf(tvm::RelayExpr const&)
        at /root/Projects/tvm-dev/src/relay/ir/expr_functor.cc:81
  42: tvm::relay::MixedModeMutator::DispatchVisitExpr(tvm::RelayExpr const&)
        at /root/Projects/tvm-dev/src/relay/ir/expr_functor.cc:94
  41: tvm::relay::ExprMutator::VisitExpr(tvm::RelayExpr const&)
        at /root/Projects/tvm-dev/src/relay/ir/expr_functor.cc:158
  40: tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>::VisitExpr(tvm::RelayExpr const&)                                                                                                                                                                                 at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:95
  39: tvm::NodeFunctor<tvm::RelayExpr (tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*)>::operator()(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*) const                           at /root/Projects/tvm-dev/include/tvm/node/functor.h:97
  38: tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>::InitVTable()::{lambda(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*)#5}::__invoke(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*)
        at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:127                                                                                                                                                                                                            37: tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>::InitVTable()::{lambda(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*)#5}::operator()(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::Relay
Expr (tvm::RelayExpr const&)>*) const                                                                                                                                                                                                                                                   at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:127
  36: tvm::relay::ExprMutator::VisitExpr_(tvm::relay::FunctionNode const*)                                                                                                                                                                                                              at /root/Projects/tvm-dev/src/relay/ir/expr_functor.cc:204
  35: tvm::relay::ExprMutator::Mutate(tvm::RelayExpr const&)                                                                                                                                                                                                                            at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:190
  34: tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)                                                                                                                                                                                                                    at /root/Projects/tvm-dev/src/relay/ir/expr_functor.cc:102                                                                                                                                                                                                                33: void tvm::relay::ExpandDataflow<tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3>(tvm::RelayExpr, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3)
        at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:492
  32: void tvm::relay::ExpandDataflow<tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3, tvm::relay::ExpandDataflow<tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3>(tvm::RelayExpr, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3)::{lambda(tvm::RelayExpr const&)#1}>(tvm::RelayExpr, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3, tvm::relay::ExpandDataflow<tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3>(tvm::RelayExpr, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_2, tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3)::{lambda(tvm::RelayExpr const&)#1})
        at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:462
  31: tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&)::$_3::operator()(tvm::RelayExpr const&) const
        at /root/Projects/tvm-dev/src/relay/ir/expr_functor.cc:98
  30: tvm::relay::MixedModeMutator::VisitLeaf(tvm::RelayExpr const&)
        at /root/Projects/tvm-dev/src/relay/ir/expr_functor.cc:81
  29: tvm::relay::MixedModeMutator::DispatchVisitExpr(tvm::RelayExpr const&)
        at /root/Projects/tvm-dev/src/relay/ir/expr_functor.cc:94
  28: tvm::relay::ExprMutator::VisitExpr(tvm::RelayExpr const&)
        at /root/Projects/tvm-dev/src/relay/ir/expr_functor.cc:158
  27: tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>::VisitExpr(tvm::RelayExpr const&)
        at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:95
  26: tvm::NodeFunctor<tvm::RelayExpr (tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*)>::operator()(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*) const
        at /root/Projects/tvm-dev/include/tvm/node/functor.h:97
  25: tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>::InitVTable()::{lambda(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*)#6}::__invoke(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayEx
pr (tvm::RelayExpr const&)>*)
        at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:128
  24: tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>::InitVTable()::{lambda(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*)#6}::operator()(tvm::runtime::ObjectRef const&, tvm::relay::ExprFunctor<tvm::Relay
Expr (tvm::RelayExpr const&)>*) const
        at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:128
  23: tvm::relay::MixedModeMutator::VisitExpr_(tvm::relay::CallNode const*)
        at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:285
  22: tvm::RelayExpr tvm::relay::MixedModeMutator::Rewrite<tvm::relay::CallNode>(tvm::relay::CallNode const*)
        at /root/Projects/tvm-dev/include/tvm/relay/expr_functor.h:307
  21: tvm::relay::ForwardRewriter::Rewrite_(tvm::relay::CallNode const*, tvm::RelayExpr const&)
        at /root/Projects/tvm-dev/src/relay/transforms/forward_rewrite.cc:152
  20: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void>&, tvm::runtime::ObjectRef>(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void>&, tvm::runtime::ObjectRef&&) const
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1369
  19: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:622
  18: std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>::AssignTypedLambda<tvm::RelayEx
pr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>(tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&))::{lambda(tvm::runtime::TVMArgs
 const&, tvm::runtime::TVMRetValue*)#1}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:291
  17: std::enable_if<__and_<std::is_void<void>, std::__is_invocable<tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>::AssignTypedLambda<tvm::RelayExpr (*)(tvm::relay::
Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>(tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&))::{lambda(tvm::runtime::TVMArgs const&, tvm::runti
me::TVMRetValue*)#1}&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*> >::value, void>::type std::__invoke_r<void, tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>:
:AssignTypedLambda<tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>(tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&))
::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef con
st&)>::AssignTypedLambda<tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>(tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef co
nst&))::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:153
  16: void std::__invoke_impl<void, tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>::AssignTypedLambda<tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array
<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>(tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&))::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}&, tvm::runt
ime::TVMArgs, tvm::runtime::TVMRetValue*>(std::__invoke_other, tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>::AssignTypedLambda<tvm::RelayExpr (*)(tvm::relay::Call
const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>(tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&))::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::T
VMRetValue*)#1}&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60
  15: tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>::AssignTypedLambda<tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&,
 tvm::runtime::ObjectRef const&)>(tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&))::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&,
 tvm::runtime::TVMRetValue*) const
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1492
  14: void tvm::runtime::detail::unpack_call<tvm::RelayExpr, 3, tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > c
onst*, tvm::RelayExpr (* const&)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&), tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1421
  13: void tvm::runtime::detail::unpack_call_dispatcher<tvm::RelayExpr, 3, 0, tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>::run<>(std::__cxx11::basic_string<char, std::char_traits<char>, std
::allocator<char> > const*, tvm::RelayExpr (* const&)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&), tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1382
  12: void tvm::runtime::detail::unpack_call_dispatcher<tvm::RelayExpr, 2, 1, tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>::run<tvm::runtime::TVMMovableArgValueWithContext_>(std::__cxx11::ba
sic_string<char, std::char_traits<char>, std::allocator<char> > const*, tvm::RelayExpr (* const&)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&), tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*, tvm::run
time::TVMMovableArgValueWithContext_&&)
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1382
  11: void tvm::runtime::detail::unpack_call_dispatcher<tvm::RelayExpr, 1, 2, tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>::run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TV
MMovableArgValueWithContext_>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const*, tvm::RelayExpr (* const&)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&), tvm::runtime::TVMArg
s const&, tvm::runtime::TVMRetValue*, tvm::runtime::TVMMovableArgValueWithContext_&&, tvm::runtime::TVMMovableArgValueWithContext_&&)
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1382
  10: void tvm::runtime::detail::unpack_call_dispatcher<tvm::RelayExpr, 0, 3, tvm::RelayExpr (*)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)>::run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TV
MMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const*, tvm::RelayExpr (* const&)(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::ru
ntime::ObjectRef const&), tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*, tvm::runtime::TVMMovableArgValueWithContext_&&, tvm::runtime::TVMMovableArgValueWithContext_&&, tvm::runtime::TVMMovableArgValueWithContext_&&)
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1397
  9: tvm::RelayExpr tvm::relay::LayoutRewriter<tvm::relay::alter_op_layout::AlterTransformMemorizer>(tvm::relay::Call const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::ObjectRef const&)
        at /root/Projects/tvm-dev/src/relay/transforms/transform_layout.h:354
  8: tvm::relay::alter_op_layout::AlterTransformMemorizerNode::CallWithNewLayouts(tvm::relay::Call const&, tvm::Attrs, std::vector<tvm::RelayExpr, std::allocator<tvm::RelayExpr> > const&)
        at /root/Projects/tvm-dev/src/relay/transforms/alter_op_layout.cc:77
  7: tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::Attrs const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::Array<tvm::te::Tensor, void> const&, tvm::Type const&)>::operator()(tvm::Attrs const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, t
vm::runtime::Array<tvm::te::Tensor, void> const&, tvm::Type const&) const
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1498
  6: tvm::RelayExpr tvm::runtime::detail::typed_packed_call_dispatcher<tvm::RelayExpr>::run<tvm::Attrs const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::Array<tvm::te::Tensor, void> const&, tvm::Type const&>(tvm::runtime::PackedFunc const&, tvm::Attr
s const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::Array<tvm::te::Tensor, void> const&, tvm::Type const&)
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1444
  5: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::Attrs const&, tvm::runtime::Array<tvm::RelayExpr, void> const&, tvm::runtime::Array<tvm::te::Tensor, void> const&, tvm::Type const&>(tvm::Attrs const&, tvm::runtime::Array<tvm::RelayExpr, void> const
&, tvm::runtime::Array<tvm::te::Tensor, void> const&, tvm::Type const&) const
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1369
  4: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:622
  3: std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), TVMFuncCreateFromCFunc::$_2>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:291
  2: std::enable_if<__and_<std::is_void<void>, std::__is_invocable<TVMFuncCreateFromCFunc::$_2&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*> >::value, void>::type std::__invoke_r<void, TVMFuncCreateFromCFunc::$_2&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>
(TVMFuncCreateFromCFunc::$_2&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:153
  1: void std::__invoke_impl<void, TVMFuncCreateFromCFunc::$_2&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(std::__invoke_other, TVMFuncCreateFromCFunc::$_2&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60
  0: TVMFuncCreateFromCFunc::$_2::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /root/Projects/tvm-dev/src/runtime/c_runtime_api.cc:526
  File "/root/Projects/tvm-dev/python/tvm/_ffi/_ctypes/packed_func.py", line 81, in cfun
    rv = local_pyfunc(*pyargs)
  File "/root/Projects/tvm-dev/python/tvm/relay/op/nn/_nn.py", line 231, in alter_op_layout_conv2d
    return topi.nn.conv2d_alter_layout(attrs, inputs, tinfos, out_type)
  File "/root/miniconda/envs/python38/lib/python3.8/site-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/root/Projects/tvm-dev/python/tvm/target/generic_func.py", line 286, in dispatch_func
    return dispatch_dict[k](*args, **kwargs)
  File "/root/Projects/tvm-dev/python/tvm/topi/cuda/conv2d_alter_op.py", line 49, in _alter_conv2d_layout
    impl, outs = relay.backend.te_compiler.select_implementation(
  File "/root/Projects/tvm-dev/python/tvm/relay/backend/te_compiler.py", line 173, in select_implementation
    all_impls = get_valid_implementations(op, attrs, inputs, out_type, target)
  File "/root/Projects/tvm-dev/python/tvm/relay/backend/te_compiler.py", line 114, in get_valid_implementations
    strategy = fstrategy(attrs, inputs, out_type, target)
  File "/root/Projects/tvm-dev/python/tvm/target/generic_func.py", line 46, in __call__
    return _ffi_api.GenericFuncCallFunc(self, *args)
  File "/root/Projects/tvm-dev/python/tvm/_ffi/_ctypes/packed_func.py", line 237, in __call__
    raise get_last_ffi_error()
  13: TVMFuncCall
  12: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /root/Projects/tvm-dev/src/runtime/c_runtime_api.cc:475
  11: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1151
  10: std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::$_5>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:622
  9: std::enable_if<__and_<std::is_void<void>, std::__is_invocable<tvm::$_5&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*> >::value, void>::type std::__invoke_r<void, tvm::$_5&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(tvm::$_5&, tvm::runtime::TVMArgs&&, t
vm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:291
  8: void std::__invoke_impl<void, tvm::$_5&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(std::__invoke_other, tvm::$_5&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:153
  7: tvm::$_5::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60
  6: tvm::GenericFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /root/Projects/tvm-dev/src/target/generic_func.cc:172
  5: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /root/Projects/tvm-dev/src/target/generic_func.cc:118
  4: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /root/Projects/tvm-dev/include/tvm/runtime/packed_func.h:1151
  3: std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), TVMFuncCreateFromCFunc::$_2>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:622
  2: std::enable_if<__and_<std::is_void<void>, std::__is_invocable<TVMFuncCreateFromCFunc::$_2&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*> >::value, void>::type std::__invoke_r<void, TVMFuncCreateFromCFunc::$_2&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>
(TVMFuncCreateFromCFunc::$_2&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:291
  1: void std::__invoke_impl<void, TVMFuncCreateFromCFunc::$_2&, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*>(std::__invoke_other, TVMFuncCreateFromCFunc::$_2&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:153
  0: TVMFuncCreateFromCFunc::$_2::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
        at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60
  File "/root/Projects/tvm-dev/python/tvm/_ffi/_ctypes/packed_func.py", line 81, in cfun
        at /root/Projects/tvm-dev/src/runtime/c_runtime_api.cc:526
    rv = local_pyfunc(*pyargs)
  File "/root/Projects/tvm-dev/python/tvm/relay/op/strategy/cuda.py", line 398, in conv2d_strategy_cuda
    raise RuntimeError("Unsupported group_conv2d layout {}".format(layout))
RuntimeError: Unsupported group_conv2d layout NHWC
0 task(s) extracted

@junrushao
Copy link
Owner Author

I'm able to confirm that the same ResNeXt fails with TenSet. Skipped

@junrushao junrushao merged commit 8e3df1b into meta-schedule Feb 14, 2022
junrushao added a commit that referenced this pull request Feb 20, 2022
@junrushao @Hzfengsy
@spectrometerHBH @jinhongyii @MasterJH5574 @vinx13 @sunggg @zxybazh
Squashed commit
1d8cc54 
[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (apache#485)

[Meta Schedule][M3c] PostOrderApply (apache#486)

Fix Post Order Apply (apache#490)

[MetaSchedule] Relay Integration (apache#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (apache#492)

Fix replay trace. (apache#493)

[M3c][Meta Schedule] Implement the Replay Func class. (apache#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (apache#494)

[Meta Schedule Refactor] Get child blocks (apache#500)

Read-at && Write-at (apache#497)

[M3c][Meta Schedule] Measure Callbacks (apache#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (apache#496)

[MetaSchedule] Sample-Perfect-Tile (apache#501)

[MetaSchedule] TE Workloads (apache#502)

[TensorIR] GetProducer, GetConsumer (apache#506)

[MetaScheduleRefactor] Annotate&Unannotate (apache#505)

[MetaSchedule] Multi-Level-Tiling & Auto-Inline (apache#503)

[Tests] Add unittests for auto-inline and multi-level-tiling (apache#508)

[Meta Schedule] Minor Fixes (apache#507)

[MetaSchedule] Rewrite Cooperative-Fetching / Unbound-Block / Reduction-Block (apache#509)

[MetaSchedule] Rewrite Parallel-Vectorize-Unroll / Verify-GPU / Disallow-Dynamic-Loops (apache#499)

[Meta Schedule] Add Helper Function & Minor Modification (apache#512)

[MetaSchedule] Test for Rewrite Parallel-Vectorize-Unroll  (apache#513)

[Meta Schedule] Feature Extractor & Cost Model (apache#510)

Blockize & Tensorize (apache#514)

Layout Rewriting: Suggest-Index-Map (apache#520)

[MetaSchedule] Parallel-Vectorize-Unroll & Random-Compute-Location (apache#516)

[Meta Schedule] Per-Store-Feature (apache#521)

Add traced schedule for blockize & tensorize (apache#526)

[Meta Schedule] Add XGBoost Model & Random Model (apache#519)

User-Interface: Tune-TIR (apache#525)

User-Interface: Tune-TE (apache#527)

[Minor] More logging on python (apache#528)

Get CUDA tuning working (apache#529)

[MetaSchedule] TensorRT BYOC (apache#518)

[BugFix] LocalBuilder API (apache#531)

[Meta Schedule] Add Cost Model Update Measure Callback (apache#530)

[Bugfix] BuilderInput with default params (apache#532)

[MetaSchedule] Mutator-Tile-Size, Mutate-Parallel, Mutate-Unroll (apache#534)

[Meta Schedule] Evolutionary Search (apache#522)

[BugFix] Remove duplicated definition of MakeMultinomialSampler (apache#535)

[Meta Schedule] Fix some bugs (apache#537)

Initiate Experiments for CPU Performance Alignment with Ansor (apache#538)

[Meta Schedule] Tweak experiment scripts (apache#539)

[Meta Schedule] Initiate experiments on CUDA (apache#540)

[TIR][Schedule] Buffer transform (apache#523)

Auto Tensor Core (apache#524)

Working on Evo Search (apache#542)

[Meta Schedule] Add Replay Tuning Interface (apache#543)

Evolutionary Search on CPU (apache#544)

Misc improvement over the error message (apache#545)

[TIR][Schedule] Software pipelining (apache#533)

[Meta Schedule Refactor] fixing unit tests (apache#547)

[MetaSchedule] Mutator-Compute-Location (apache#548)

Misc Improvement of Evolutionary Search (apache#549)

Hotfix for software pipeline (apache#552)

Misc Improvement (apache#550)

[Cherry-Pick][TensorIR] Primitive "SetScope" (apache#9738) (apache#555)

Rule RFactor (apache#551)

[MemHammer] Rewrite Rules (apache#554)

[MetaSchedule] Schedule Rule: Cross-Thread Reduction (apache#556)

[MetaSchedule] Performance Alignment - NRM and SFM (CUDA) (apache#559)

[MetaSchedule] Perf Alignment - NRM on CUDA (apache#560)

[TIR] Reorder the block iters of the blocks generated by RFactor (apache#561)

Removing 2 unit tests for software pipelining (apache#562)

[MemHammer] Lower Pass + Unittests (apache#557)

Perf Align: Remove Auto-inline before Multi-level-tiling (apache#564)

Fix Sketch Generation Unittests (apache#565)

speed up VerifyGpuCode (apache#568)

[Performance Align] fixing codegen problems (apache#569)

[Meta schedule] improve search space (#1)

Hot fix for bound predicate (#3)

[Meta Schedule] Update Tune Relay (#4)

[Performance Align] fixing codegen problems (#5)

[PerfAlign] NRM & SFM on Raspi Aligned (#6)

[BugFix] Apply bound predicate directly to loops when possible (#12)

[BugFix] Fix CrossThreadReduction on CUDA (#13)

[MetaSchedule] Enable BertTuning with MetaScheduler (#11)

[Minor][MemHammer] Minor tweaks in code review (#14)

[Meta Schedule] Add customizable search space to PostOrderApply. (#16)

Fix cooperative fetching (#17)

Fixes for codegen (#18)

[Hotfix] A unittest (#19)

Fix for GRP sketch gen (#21)

Add threadIdx filtering in Multi-Level-Tiling and Verify-GPU-Code (#20)

[BugFix][TIR] Fix cross-thread reduction when single reduction loop with predicate (apache#10016) (#22)

[MemHammer][Refactor] Code Review (#15)

[Meta Schedule] Add Winograd Test for Customizable Search Space (#24)

Import & Cache Mechanism (#26)

[BugFix] Fix Winograd Test Script (#25)

Add task extraction & caching (#27)

A few fixes for task extraction (#28)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
junrushao added a commit that referenced this pull request Feb 20, 2022
@junrushao @Hzfengsy
@spectrometerHBH @jinhongyii @MasterJH5574 @vinx13 @sunggg @zxybazh
Squashed commit
893555f 
[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (apache#485)

[Meta Schedule][M3c] PostOrderApply (apache#486)

Fix Post Order Apply (apache#490)

[MetaSchedule] Relay Integration (apache#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (apache#492)

Fix replay trace. (apache#493)

[M3c][Meta Schedule] Implement the Replay Func class. (apache#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (apache#494)

[Meta Schedule Refactor] Get child blocks (apache#500)

Read-at && Write-at (apache#497)

[M3c][Meta Schedule] Measure Callbacks (apache#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (apache#496)

[MetaSchedule] Sample-Perfect-Tile (apache#501)

[MetaSchedule] TE Workloads (apache#502)

[TensorIR] GetProducer, GetConsumer (apache#506)

[MetaScheduleRefactor] Annotate&Unannotate (apache#505)

[MetaSchedule] Multi-Level-Tiling & Auto-Inline (apache#503)

[Tests] Add unittests for auto-inline and multi-level-tiling (apache#508)

[Meta Schedule] Minor Fixes (apache#507)

[MetaSchedule] Rewrite Cooperative-Fetching / Unbound-Block / Reduction-Block (apache#509)

[MetaSchedule] Rewrite Parallel-Vectorize-Unroll / Verify-GPU / Disallow-Dynamic-Loops (apache#499)

[Meta Schedule] Add Helper Function & Minor Modification (apache#512)

[MetaSchedule] Test for Rewrite Parallel-Vectorize-Unroll  (apache#513)

[Meta Schedule] Feature Extractor & Cost Model (apache#510)

Blockize & Tensorize (apache#514)

Layout Rewriting: Suggest-Index-Map (apache#520)

[MetaSchedule] Parallel-Vectorize-Unroll & Random-Compute-Location (apache#516)

[Meta Schedule] Per-Store-Feature (apache#521)

Add traced schedule for blockize & tensorize (apache#526)

[Meta Schedule] Add XGBoost Model & Random Model (apache#519)

User-Interface: Tune-TIR (apache#525)

User-Interface: Tune-TE (apache#527)

[Minor] More logging on python (apache#528)

Get CUDA tuning working (apache#529)

[MetaSchedule] TensorRT BYOC (apache#518)

[BugFix] LocalBuilder API (apache#531)

[Meta Schedule] Add Cost Model Update Measure Callback (apache#530)

[Bugfix] BuilderInput with default params (apache#532)

[MetaSchedule] Mutator-Tile-Size, Mutate-Parallel, Mutate-Unroll (apache#534)

[Meta Schedule] Evolutionary Search (apache#522)

[BugFix] Remove duplicated definition of MakeMultinomialSampler (apache#535)

[Meta Schedule] Fix some bugs (apache#537)

Initiate Experiments for CPU Performance Alignment with Ansor (apache#538)

[Meta Schedule] Tweak experiment scripts (apache#539)

[Meta Schedule] Initiate experiments on CUDA (apache#540)

[TIR][Schedule] Buffer transform (apache#523)

Auto Tensor Core (apache#524)

Working on Evo Search (apache#542)

[Meta Schedule] Add Replay Tuning Interface (apache#543)

Evolutionary Search on CPU (apache#544)

Misc improvement over the error message (apache#545)

[TIR][Schedule] Software pipelining (apache#533)

[Meta Schedule Refactor] fixing unit tests (apache#547)

[MetaSchedule] Mutator-Compute-Location (apache#548)

Misc Improvement of Evolutionary Search (apache#549)

Hotfix for software pipeline (apache#552)

Misc Improvement (apache#550)

[Cherry-Pick][TensorIR] Primitive "SetScope" (apache#9738) (apache#555)

Rule RFactor (apache#551)

[MemHammer] Rewrite Rules (apache#554)

[MetaSchedule] Schedule Rule: Cross-Thread Reduction (apache#556)

[MetaSchedule] Performance Alignment - NRM and SFM (CUDA) (apache#559)

[MetaSchedule] Perf Alignment - NRM on CUDA (apache#560)

[TIR] Reorder the block iters of the blocks generated by RFactor (apache#561)

Removing 2 unit tests for software pipelining (apache#562)

[MemHammer] Lower Pass + Unittests (apache#557)

Perf Align: Remove Auto-inline before Multi-level-tiling (apache#564)

Fix Sketch Generation Unittests (apache#565)

speed up VerifyGpuCode (apache#568)

[Performance Align] fixing codegen problems (apache#569)

[Meta schedule] improve search space (#1)

Hot fix for bound predicate (#3)

[Meta Schedule] Update Tune Relay (#4)

[Performance Align] fixing codegen problems (#5)

[PerfAlign] NRM & SFM on Raspi Aligned (#6)

[BugFix] Apply bound predicate directly to loops when possible (#12)

[BugFix] Fix CrossThreadReduction on CUDA (#13)

[MetaSchedule] Enable BertTuning with MetaScheduler (#11)

[Minor][MemHammer] Minor tweaks in code review (#14)

[Meta Schedule] Add customizable search space to PostOrderApply. (#16)

Fix cooperative fetching (#17)

Fixes for codegen (#18)

[Hotfix] A unittest (#19)

Fix for GRP sketch gen (#21)

Add threadIdx filtering in Multi-Level-Tiling and Verify-GPU-Code (#20)

[BugFix][TIR] Fix cross-thread reduction when single reduction loop with predicate (apache#10016) (#22)

[MemHammer][Refactor] Code Review (#15)

[Meta Schedule] Add Winograd Test for Customizable Search Space (#24)

Import & Cache Mechanism (#26)

[BugFix] Fix Winograd Test Script (#25)

Add task extraction & caching (#27)

A few fixes for task extraction (#28)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
junrushao added a commit that referenced this pull request Feb 20, 2022
@junrushao @Hzfengsy
@spectrometerHBH @jinhongyii @MasterJH5574 @vinx13 @sunggg @zxybazh
Squashed commit
46a98f1 
[Meta Schedule][M3c] Schedule Rules, Mutator & Postprocs (apache#485)

[Meta Schedule][M3c] PostOrderApply (apache#486)

Fix Post Order Apply (apache#490)

[MetaSchedule] Relay Integration (apache#489)

[M3c][Meta Schedule] Add Trace Correctness Test for PostOrderApply (apache#492)

Fix replay trace. (apache#493)

[M3c][Meta Schedule] Implement the Replay Func class. (apache#495)

[PR] Test script for meta-schedule task extraction. Interface to load… (apache#494)

[Meta Schedule Refactor] Get child blocks (apache#500)

Read-at && Write-at (apache#497)

[M3c][Meta Schedule] Measure Callbacks (apache#498)

[Bug] Fix Infinite Loop Caused When Calling Methods Not Overrided In PyClass (apache#496)

[MetaSchedule] Sample-Perfect-Tile (apache#501)

[MetaSchedule] TE Workloads (apache#502)

[TensorIR] GetProducer, GetConsumer (apache#506)

[MetaScheduleRefactor] Annotate&Unannotate (apache#505)

[MetaSchedule] Multi-Level-Tiling & Auto-Inline (apache#503)

[Tests] Add unittests for auto-inline and multi-level-tiling (apache#508)

[Meta Schedule] Minor Fixes (apache#507)

[MetaSchedule] Rewrite Cooperative-Fetching / Unbound-Block / Reduction-Block (apache#509)

[MetaSchedule] Rewrite Parallel-Vectorize-Unroll / Verify-GPU / Disallow-Dynamic-Loops (apache#499)

[Meta Schedule] Add Helper Function & Minor Modification (apache#512)

[MetaSchedule] Test for Rewrite Parallel-Vectorize-Unroll  (apache#513)

[Meta Schedule] Feature Extractor & Cost Model (apache#510)

Blockize & Tensorize (apache#514)

Layout Rewriting: Suggest-Index-Map (apache#520)

[MetaSchedule] Parallel-Vectorize-Unroll & Random-Compute-Location (apache#516)

[Meta Schedule] Per-Store-Feature (apache#521)

Add traced schedule for blockize & tensorize (apache#526)

[Meta Schedule] Add XGBoost Model & Random Model (apache#519)

User-Interface: Tune-TIR (apache#525)

User-Interface: Tune-TE (apache#527)

[Minor] More logging on python (apache#528)

Get CUDA tuning working (apache#529)

[MetaSchedule] TensorRT BYOC (apache#518)

[BugFix] LocalBuilder API (apache#531)

[Meta Schedule] Add Cost Model Update Measure Callback (apache#530)

[Bugfix] BuilderInput with default params (apache#532)

[MetaSchedule] Mutator-Tile-Size, Mutate-Parallel, Mutate-Unroll (apache#534)

[Meta Schedule] Evolutionary Search (apache#522)

[BugFix] Remove duplicated definition of MakeMultinomialSampler (apache#535)

[Meta Schedule] Fix some bugs (apache#537)

Initiate Experiments for CPU Performance Alignment with Ansor (apache#538)

[Meta Schedule] Tweak experiment scripts (apache#539)

[Meta Schedule] Initiate experiments on CUDA (apache#540)

[TIR][Schedule] Buffer transform (apache#523)

Auto Tensor Core (apache#524)

Working on Evo Search (apache#542)

[Meta Schedule] Add Replay Tuning Interface (apache#543)

Evolutionary Search on CPU (apache#544)

Misc improvement over the error message (apache#545)

[TIR][Schedule] Software pipelining (apache#533)

[Meta Schedule Refactor] fixing unit tests (apache#547)

[MetaSchedule] Mutator-Compute-Location (apache#548)

Misc Improvement of Evolutionary Search (apache#549)

Hotfix for software pipeline (apache#552)

Misc Improvement (apache#550)

[Cherry-Pick][TensorIR] Primitive "SetScope" (apache#9738) (apache#555)

Rule RFactor (apache#551)

[MemHammer] Rewrite Rules (apache#554)

[MetaSchedule] Schedule Rule: Cross-Thread Reduction (apache#556)

[MetaSchedule] Performance Alignment - NRM and SFM (CUDA) (apache#559)

[MetaSchedule] Perf Alignment - NRM on CUDA (apache#560)

[TIR] Reorder the block iters of the blocks generated by RFactor (apache#561)

Removing 2 unit tests for software pipelining (apache#562)

[MemHammer] Lower Pass + Unittests (apache#557)

Perf Align: Remove Auto-inline before Multi-level-tiling (apache#564)

Fix Sketch Generation Unittests (apache#565)

speed up VerifyGpuCode (apache#568)

[Performance Align] fixing codegen problems (apache#569)

[Meta schedule] improve search space (#1)

Hot fix for bound predicate (#3)

[Meta Schedule] Update Tune Relay (#4)

[Performance Align] fixing codegen problems (#5)

[PerfAlign] NRM & SFM on Raspi Aligned (#6)

[BugFix] Apply bound predicate directly to loops when possible (#12)

[BugFix] Fix CrossThreadReduction on CUDA (#13)

[MetaSchedule] Enable BertTuning with MetaScheduler (#11)

[Minor][MemHammer] Minor tweaks in code review (#14)

[Meta Schedule] Add customizable search space to PostOrderApply. (#16)

Fix cooperative fetching (#17)

Fixes for codegen (#18)

[Hotfix] A unittest (#19)

Fix for GRP sketch gen (#21)

Add threadIdx filtering in Multi-Level-Tiling and Verify-GPU-Code (#20)

[BugFix][TIR] Fix cross-thread reduction when single reduction loop with predicate (apache#10016) (#22)

[MemHammer][Refactor] Code Review (#15)

[Meta Schedule] Add Winograd Test for Customizable Search Space (#24)

Import & Cache Mechanism (#26)

[BugFix] Fix Winograd Test Script (#25)

Add task extraction & caching (#27)

A few fixes for task extraction (#28)

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Sunghyun Park <49998730+sunggg@users.noreply.github.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@junrushao