Disable pool&conv_transpose&quantize caching #36695

jczaja · 2021-10-25T08:56:13Z

PR types

Bug fixes

PR changes

OPs

Describe

PaddlePaddle is caching oneDNN objects and this PR is to disable this for pool2d and conv_transpose. Reason is that oneDNN is caching some of its objects itself

paddle-bot-old · 2021-10-25T08:56:16Z

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

jczaja · 2021-10-25T13:11:33Z

@wozna Please take a look at changes in quantize.

jczaja · 2021-10-26T08:59:07Z

@tsocha Please review

jakpiase · 2021-10-26T12:25:52Z

paddle/fluid/operators/mkldnn/conv_transpose_mkldnn_op.cc

+        for (int j = 0; j < o; ++j) {
+          int in_offset = j * h * w + i * o * h * w;
+          int out_offset = j * c * h * w + i * h * w;
+          std::memcpy(&(reordered_filter_data.get())[out_offset],


I know that might not be your code, but shouldn't that reorder be done using oneDNN?

:) @Silv3S is working on that.

lidanqing-intel

LGTM

jczaja · 2021-10-26T17:03:39Z

@chenwhql Could you please approve so that PR-CI-APPROVAL pass? PADDLE_ENFORCE is correct but due only part of it is visible in diff, corresponding checker cannot see full statement and report error.

lidanqing-intel · 2021-10-27T01:47:05Z

@baoachun @Superjomn This PR is part of the improving cache mechanism. CI reported PADDLE_ENFORCE error but actually it is correct but due only part of it is visible in diff, corresponding checker cannot see full statement and report error.
Could you please help approve it? Thanks.

lidanqing-intel · 2021-10-28T08:23:52Z

paddle/fluid/operators/mkldnn/conv_transpose_mkldnn_op.cc

                        platform::errors::InvalidArgument(
-                            "Got wrong formats for Filter tensor."));
+                            "Got wrong format for Bias tensor."));


Hi, @baoachun
这里，错误信息长度符合要求，要求至少20个字符长度，这里错误信息有30个字符左右长度。这里的问题是PADDLE_ENFORCE 机制不认未更改的行，这里 95行 platform::errors::InvalidArgument(已经写了，只是这个PR没有更改，所以就没检测到，图片里InvalidArgument代码就没有。但是其实代码中是有的，是PADLLE_ENFORCE机制没检测到。

这个PADDLE_ENFORCE机制以后会考虑优化吗，以后可能也经常有类似问题。

这里是字符串正则匹配检测的，先grep找出修改了的行，然后正则匹配做合规检查，如果全文件正则检查，可以避免这个问题，但是可能会检出和PR不相关的一些问题，不太友好。。。目前这一点没有想到好的办法处理，先找我approve吧

好的，非常感谢！

wozna

LGTM

jakpiase

LGTM

paddle-bot-old · 2021-11-02T02:34:53Z

Sorry to inform you that b27d1b8's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

chenwhql

LGTM for PADDLE_ENFORCE

lidanqing-intel · 2021-11-02T04:58:14Z

@jczaja This PR is approved by Baidu. But could you please watch PR-CI-iScan-C to pass ? Cause only this one keep failing.

jczaja · 2021-11-02T12:15:37Z

@baoachun Please help with failing CI (it did pass few reruns ago). Error is: "git code download failed ". I have try couple of reruns but without success.

baoachun · 2021-11-03T03:04:06Z

@baoachun Please help with failing CI (it did pass few reruns ago). Error is: "git code download failed ". I have try couple of reruns but without success.

My colleague is solving the problem, please wait.

baoachun · 2021-11-04T04:30:58Z

Hi @jczaja, my colleague said that you may need to pull the latest code of the develop branch.

- compilation fix - fix - fixes - fix - fix - fix again - fix - another fix - another compilation fix - fix - fix - fix - lint

- pool2d partially stripped of caching

jczaja · 2021-11-04T08:56:32Z

@baoachun I have rebased this PR but still PR-CI-iScan-C is failing (I have done one rerurn). Please help

jczaja added the Intel label Oct 25, 2021

jczaja changed the title ~~[WIP] disable pool&conv_transpose caching~~ Disable pool&conv_transpose&quantize caching Oct 25, 2021

jczaja requested review from wozna and jakpiase October 25, 2021 13:10

tsocha previously approved these changes Oct 26, 2021

View reviewed changes

jakpiase reviewed Oct 26, 2021

View reviewed changes

lidanqing-intel self-requested a review October 26, 2021 16:10

lidanqing-intel previously approved these changes Oct 26, 2021

View reviewed changes

lidanqing-intel reviewed Oct 28, 2021

View reviewed changes

wozna previously approved these changes Oct 28, 2021

View reviewed changes

jakpiase previously approved these changes Oct 28, 2021

View reviewed changes

jczaja assigned chenwhql Oct 29, 2021

lidanqing-intel changed the title ~~Disable pool&conv_transpose&quantize caching~~ [Approval Needed] Disable pool&conv_transpose&quantize caching Nov 1, 2021

chenwhql previously approved these changes Nov 2, 2021

View reviewed changes

lidanqing-intel changed the title ~~[Approval Needed] Disable pool&conv_transpose&quantize caching~~ [Approval-CI Passed] Disable pool&conv_transpose&quantize caching Nov 2, 2021

lidanqing-intel changed the title ~~[Approval-CI Passed] Disable pool&conv_transpose&quantize caching~~ Disable pool&conv_transpose&quantize caching Nov 2, 2021

lidanqing-intel changed the title ~~Disable pool&conv_transpose&quantize caching~~ [Approved] Disable pool&conv_transpose&quantize caching Nov 2, 2021

jczaja added 15 commits November 4, 2021 09:10

- WIP

57768e5

- compilation fix - fix - fixes - fix - fix - fix again - fix - another fix - another compilation fix - fix - fix - fix - lint

- pool2d partially stripped from cache

d94cf97

- pool2d partially stripped of caching

- compilation fix

ca087d7

- compilation fix

dfb04fa

- Fix to UT of caching

8b39ee8

- Enabling test_conv3d_mkldnn

e076e3d

- conv_transpose stripped of cache

23fa191

- compilation fix

43a945f

- fix

24b3a4b

- fix

f9bfcc4

- compilation fix

d5de5ee

- fix

d57ce1d

Reverted disabling caching of conv2d

4771b9d

- compilation fix

cb0f692

- ut reverted

8567968

jczaja dismissed stale reviews from chenwhql, jakpiase, wozna, lidanqing-intel, and tsocha via 8567968 November 4, 2021 08:12

jczaja force-pushed the prv-disable-pool-caching branch from b27d1b8 to 8567968 Compare November 4, 2021 08:12

lidanqing-intel changed the title ~~[Approved] Disable pool&conv_transpose&quantize caching~~ [Waiting some CIs exemption] Disable pool&conv_transpose&quantize caching Nov 4, 2021

chenwhql approved these changes Nov 5, 2021

View reviewed changes

lidanqing-intel changed the title ~~[Waiting some CIs exemption] Disable pool&conv_transpose&quantize caching~~ Disable pool&conv_transpose&quantize caching Nov 5, 2021

jczaja merged commit db6c00c into PaddlePaddle:develop Nov 5, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Disable pool&conv_transpose&quantize caching #36695

Disable pool&conv_transpose&quantize caching #36695

jczaja commented Oct 25, 2021

paddle-bot-old bot commented Oct 25, 2021

jczaja commented Oct 25, 2021

jczaja commented Oct 26, 2021

jakpiase Oct 26, 2021 •

edited

Loading

jczaja Oct 26, 2021

lidanqing-intel left a comment

jczaja commented Oct 26, 2021

lidanqing-intel commented Oct 27, 2021

lidanqing-intel Oct 28, 2021 •

edited

Loading

lidanqing-intel Oct 28, 2021

lidanqing-intel Oct 28, 2021

chenwhql Nov 2, 2021

lidanqing-intel Nov 2, 2021

wozna left a comment

jakpiase left a comment

paddle-bot-old bot commented Nov 2, 2021

chenwhql left a comment

lidanqing-intel commented Nov 2, 2021

jczaja commented Nov 2, 2021

baoachun commented Nov 3, 2021

baoachun commented Nov 4, 2021

jczaja commented Nov 4, 2021

Disable pool&conv_transpose&quantize caching #36695

Disable pool&conv_transpose&quantize caching #36695

Conversation

jczaja commented Oct 25, 2021

PR types

PR changes

Describe

paddle-bot-old bot commented Oct 25, 2021

jczaja commented Oct 25, 2021

jczaja commented Oct 26, 2021

jakpiase Oct 26, 2021 • edited Loading

Choose a reason for hiding this comment

jczaja Oct 26, 2021

Choose a reason for hiding this comment

lidanqing-intel left a comment

Choose a reason for hiding this comment

jczaja commented Oct 26, 2021

lidanqing-intel commented Oct 27, 2021

lidanqing-intel Oct 28, 2021 • edited Loading

Choose a reason for hiding this comment

lidanqing-intel Oct 28, 2021

Choose a reason for hiding this comment

lidanqing-intel Oct 28, 2021

Choose a reason for hiding this comment

chenwhql Nov 2, 2021

Choose a reason for hiding this comment

lidanqing-intel Nov 2, 2021

Choose a reason for hiding this comment

wozna left a comment

Choose a reason for hiding this comment

jakpiase left a comment

Choose a reason for hiding this comment

paddle-bot-old bot commented Nov 2, 2021

chenwhql left a comment

Choose a reason for hiding this comment

lidanqing-intel commented Nov 2, 2021

jczaja commented Nov 2, 2021

baoachun commented Nov 3, 2021

baoachun commented Nov 4, 2021

jczaja commented Nov 4, 2021

jakpiase Oct 26, 2021 •

edited

Loading

lidanqing-intel Oct 28, 2021 •

edited

Loading