Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark cmsRun as a likely candidate for the kernel OOM killer #45855

Open
fwyzard opened this issue Sep 2, 2024 · 4 comments
Open

Mark cmsRun as a likely candidate for the kernel OOM killer #45855

fwyzard opened this issue Sep 2, 2024 · 4 comments

Comments

@fwyzard
Copy link
Contributor

fwyzard commented Sep 2, 2024

On a worker node without hard memory limits, cmsRun may occasionally cause an out-of-memory (OOM) situation that leads to a system process being killed by the kernel.

The kernel can be "encouraged" to kill a cmsRun process instead of some other process setting /proc/PID/oom_score_adj to a value larger than 0, up to 1000 (see man oom_score_adj).

As this needs to be set for each process, would it make sense to let cmsRun set the value itself when it starts, writing to /proc/self/oom_score_adj ?

We could use something like process.options.oomScoreAdjust to make it configurable, and start with a default value between 100 (somewhat more likely) and 500 (much more likely).

@fwyzard
Copy link
Contributor Author

fwyzard commented Sep 2, 2024

assign core

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2024

New categories assigned: core

@Dr15Jones,@makortel,@smuzaffar you have been requested to review this Pull request/Issue and eventually sign? Thanks

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2024

cms-bot internal usage

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 2, 2024

A new Issue was created by @fwyzard.

@Dr15Jones, @antoniovilela, @makortel, @mandrenguyen, @rappoccio, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants