You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Current backward implementation is faced with several problems.
The design of current backward didn't think much about nested blocks. To make it working on WhileOp and IfElesOp, we have to add many tricks into it, which makes our code extremely hard to read.
Incomplete implementation of backward pruning. We are simply using a set to record names of variables which do not have gradients. This strategy can not disgust variables in different blocks with the same name and causes bugs in some special cases.
In some cases, such as error clipping, we need to insert some other operators(ClipOp) between all backward operators. It's not easy to implement these functions in current C++ code.
The text was updated successfully, but these errors were encountered:
Current backward implementation is faced with several problems.
The design of current backward didn't think much about nested blocks. To make it working on
WhileOp
andIfElesOp
, we have to add many tricks into it, which makes our code extremely hard to read.Incomplete implementation of backward pruning. We are simply using a
set
to record names of variables which do not have gradients. This strategy can not disgust variables in different blocks with the same name and causes bugs in some special cases.In some cases, such as error clipping, we need to insert some other operators(
ClipOp
) between all backward operators. It's not easy to implement these functions in current C++ code.The text was updated successfully, but these errors were encountered: