Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems in current backward implementation #6600

Closed
JiayiFeng opened this issue Dec 14, 2017 · 0 comments · Fixed by #6741
Closed

Problems in current backward implementation #6600

JiayiFeng opened this issue Dec 14, 2017 · 0 comments · Fixed by #6741
Assignees

Comments

@JiayiFeng
Copy link
Collaborator

JiayiFeng commented Dec 14, 2017

Current backward implementation is faced with several problems.

  1. The design of current backward didn't think much about nested blocks. To make it working on WhileOp and IfElesOp, we have to add many tricks into it, which makes our code extremely hard to read.

  2. Incomplete implementation of backward pruning. We are simply using a set to record names of variables which do not have gradients. This strategy can not disgust variables in different blocks with the same name and causes bugs in some special cases.

  3. In some cases, such as error clipping, we need to insert some other operators(ClipOp) between all backward operators. It's not easy to implement these functions in current C++ code.

@JiayiFeng JiayiFeng added this to the Release 0.11.1 milestone Dec 14, 2017
@JiayiFeng JiayiFeng self-assigned this Dec 14, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant