Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add apex support with opt level o1 #50

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Conversation

lbin
Copy link

@lbin lbin commented Feb 6, 2020

No description provided.

@JesseYang
Copy link

I followed the code in this branch and got the following error:
RuntimeError: Function _DCNv2Backward returned an invalid gradient at index 1 - expected type torch.cuda.HalfTensor but got torch.cuda.FloatTensor

@lbin
Copy link
Author

lbin commented Apr 20, 2020

https://github.com/lbin/DCNv2/blob/master/dcn_v2.py#L43 maybe you should clean you dcn env and recompile the dcn codes @JesseYang

@JesseYang
Copy link

Thanks for your reply. I don't think recompiling can solve this problem. I re-clone the code from https://github.com/lbin/DCNv2 and build it, but still got the same error. My pytorch version is 1.2. The usage of DCN in my network is:
self.dcn = DCN(input_channel, output_channel, kernel_size=(3, 3), stride=1, padding=1, dilation=1, deformable_groups=1)
......
enc = self.dcn(enc)

@JesseYang
Copy link

The problem is solved. I add the following three lines before return in the _backward function of _DCNv2 in dcn_v2.py:

grad_input = grad_input.half()
grad_offset = grad_offset.half()
grad_mask = grad_mask.half()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants