Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option to set number of warmup iterations #27739

Conversation

wojtuss
Copy link

@wojtuss wojtuss commented Sep 30, 2020

PR types

Others

PR changes

Others

Describe

With this patch one can set the number of iterations for warmup phase in performance tests. When the option --warmup=true is used, by default one iteration of inference is run as a warmup. Now, with additional option --warmup_iters=5 the given number of iterations will be run in warmup.
This option helps properly measure performance of GRU FP32/INT8 model with oneDNN. When running GRU inference with oneDNN, the oneDNN cache needs more than one (actually close to 20) iterations to cache all the primitives required during inference of the whole dataset.

@wojtuss wojtuss added the Intel label Sep 30, 2020
@paddle-bot-old
Copy link

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@wojtuss wojtuss requested a review from luotao1 October 1, 2020 06:27
Copy link
Contributor

@luotao1 luotao1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this PR make the elapsed time of the unit-test become longer since each unit-test has an extra warmup iteration? Could the default warmup_iter be 0?

@wojtuss
Copy link
Author

wojtuss commented Oct 1, 2020

Does this PR make the elapsed time of the unit-test become longer since each unit-test has an extra warmup iteration? Could the default warmup_iter be 0?

@luotao1
The option --wamup_iters only changes the number of warmup iterations if the --warmup option is used. If the --warmup option is not used, or --warmup_iters is not used or is set to a number <=1, then it has no effect (i.e. still the default number of warmup iterations (1) is used for the --warmup option). If it is set to a number >1, then yes, it has impact on execution time of the tests which use this option, because more inference iterations are executed during warmup. However, for GRU model test the warmup iterations are rather quick.

@luotao1
Copy link
Contributor

luotao1 commented Oct 1, 2020

Got it, thanks for your explanation.

@luotao1 luotao1 merged commit f399bed into PaddlePaddle:develop Oct 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants