Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix DataLoader memory leak #34140

Merged
merged 2 commits into from
Jul 21, 2021
Merged

fix DataLoader memory leak #34140

merged 2 commits into from
Jul 21, 2021

Conversation

heavengate
Copy link
Contributor

@heavengate heavengate commented Jul 13, 2021 •
edited
Loading

PR types

Bug fixes

PR changes

APIs

Describe

fix DataLoader memory leak introduced by #32550

  • register __del__ to atexit may remain a link to DataLoader object, which make DataLoader cannot be GCed when epoch finish
  • simply add __del__ to release resource can also work to fix DataLoader exit error, for __del__ will be called when DataLoader object GCed, so remove atexit register here

@paddle-bot-old
Copy link

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@PaddlePaddle PaddlePaddle locked and limited conversation to collaborators Jul 19, 2021
@PaddlePaddle PaddlePaddle unlocked this conversation Jul 19, 2021
chenwhql
chenwhql approved these changes Jul 21, 2021
View reviewed changes
Copy link
Contributor

@chenwhql chenwhql left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@heavengate heavengate mentioned this pull request Jul 21, 2021
Merged
@heavengate heavengate merged commit 6fc33a0 into develop Jul 21, 2021
@qingqing01 qingqing01 mentioned this pull request Jul 22, 2021
Closed
@yghstill yghstill mentioned this pull request Jul 23, 2021
Closed
@guoshengCS guoshengCS mentioned this pull request Aug 5, 2021
Closed
@paddle-bot-old paddle-bot-old bot deleted the fix_dataloader_memory_leak branch October 15, 2021 06:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chenwhql chenwhql chenwhql approved these changes

@LielinJiang LielinJiang Awaiting requested review from LielinJiang

@qingqing01 qingqing01 Awaiting requested review from qingqing01

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@heavengate @chenwhql