Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lower memory usage for aptly db cleanup #762

Merged
merged 1 commit into from
Jul 31, 2018
Merged

Conversation

smira
Copy link
Contributor

@smira smira commented Jul 19, 2018

Description of the Change

This is not a complete fix, but the easiest first step.

During db cleanup, aptly is loading every repo/mirror/... into memory,
and even though each object is processed only once, collection holds
a reference to all the loaded objects, so they won't be GC'd until
process exits.

CollectionFactory.Flush() releases pointers to collection objects,
making objects egligble for GC.

This is not a complete fix, as during iteration we could have tried
to release a link to every object being GCed and that would have
helped much more.

See #761

Checklist

  • unit-test added (if change is algorithm)
  • functional test added/updated (if change is functional)
  • man page updated (if applicable)
  • bash completion updated (if applicable)
  • documentation updated
  • author name in AUTHORS

This is not a complete fix, but the easiest first step.

During `db cleanup`, aptly is loading every repo/mirror/... into memory,
and even though each object is processed only once, collection holds
a reference to all the loaded objects, so they won't be GC'd until
process exits.

CollectionFactory.Flush() releases pointers to collection objects,
making objects egligble for GC.

This is not a complete fix, as during iteration we could have tried
to release a link to every object being GCed and that would have
helped much more.
@smira smira added this to the 1.4.0 milestone Jul 19, 2018
@codecov
Copy link

codecov bot commented Jul 20, 2018

Codecov Report

Merging #762 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@          Coverage Diff           @@
##           master    #762   +/-   ##
======================================
  Coverage    63.7%   63.7%           
======================================
  Files          50      50           
  Lines        6271    6271           
======================================
  Hits         3995    3995           
  Misses       1788    1788           
  Partials      488     488

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bcacb7b...021b8c4. Read the comment docs.

@smira smira requested a review from a team July 20, 2018 14:45
Copy link
Contributor

@sliverc sliverc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change LGTM.

I guess it might be a good idea whether this could be tested in a real life scenario though before this is merged or maybe released whether it really improves the situation.

@smira
Copy link
Contributor Author

smira commented Jul 31, 2018

I will merge this PR and I will start working on a more complete solution at the same time.

@smira smira merged commit 86a1c41 into master Jul 31, 2018
@sliverc sliverc deleted the 761-flush-collections branch August 1, 2018 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants