Changed IsCriticalPod to return true in case of static pods #80491

hpandeycodeit · 2019-07-23T21:03:27Z

What type of PR is this?
/kind bug

What this PR does / why we need it:

Currently, If the static pods is critical, the kubelet will not realize it, since it's priority value determines whether the pod is critical or not. So SIG Node decided to consider all static pods as critical regardless of their PodSpec.PriorityClassName.

Which issue(s) this PR fixes:

Fixes #80489

Does this PR introduce a user-facing change?:

Kubelet considers all static pods as critical. Static pods pass kubelet admission even if a node does not have enough resources. Users must ensure that they account for resources when creating static pods.

hpandeycodeit · 2019-07-23T21:04:03Z

/sig scheduling
/sig node

hpandeycodeit · 2019-07-23T21:50:29Z

/retest

draveness

/priority important-soon

mattjmcnaughton

Thanks for this pr :)

I'm curious to see the results of the test suite after running the two hack scripts below. You may have an import cycle, in which case we may need to rethink the approach slightly. We'll know for sure after re-running the test suite after running the hack scripts.

mattjmcnaughton · 2019-07-24T13:39:31Z

pkg/kubelet/types/pod_update.go

@@ -23,6 +23,7 @@ import (
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	utilfeature "k8s.io/apiserver/pkg/util/feature"
 	kubeapi "k8s.io/kubernetes/pkg/apis/core"
+	kubepod "k8s.io/kubernetes/pkg/kubelet/pod"


Please run ./hack/update-bazel.sh to add this new dependency to the proper BUILD.bazel file.

I ran the above hack scripts. Doesn't look like it's helping with the import cycle though :(

mattjmcnaughton · 2019-07-24T13:40:17Z

pkg/kubelet/types/pod_update.go

@@ -146,6 +147,9 @@ func (sp SyncPodType) String() string {
 // or equal to SystemCriticalPriority. Both the default scheduler and the kubelet use this function
 // to make admission and scheduling decisions.
 func IsCriticalPod(pod *v1.Pod) bool {
+	if kubepod.IsStaticPod(pod){


Please run ./hack/update-gofmt.sh :)

bsalamat

Looks good. You may have to move IsCriticalPod to pkg/kubelet/pod, or another similar move to remove the circular dependency.
Also, please make sure to add adequate tests to cover the new logic.

hpandeycodeit · 2019-07-24T22:18:48Z

circular dependency

Thanks @bsalamat! I made the needed change to remove the import cycle.

bsalamat · 2019-07-25T00:53:40Z

pkg/kubelet/types/pod_update_test.go

+					Name:      "pod5",
+					Namespace: "kube-system",
+					Annotations: map[string]string{
+						"scheduler.alpha.kubernetes.io/critical-pod": "file",


If you meant to add a static pod, the key should have been ConfigSourceAnnotationKey. What you have used is the deprecated critical annotation key.

hpandeycodeit · 2019-07-25T15:40:04Z

/test pull-kubernetes-e2e-gce-100-performance

bsalamat

/lgtm

Please go ahead and squash commits.
Thanks, @hpandeycodeit!

hpandeycodeit · 2019-07-29T20:30:47Z

/test pull-kubernetes-e2e-gce-device-plugin-gpu

bsalamat

/lgtm

/assign @dchen1107
for reviewing and approval

derekwaynecarr · 2019-08-06T14:47:06Z

I am not necessarily adverse to this. I know I was out of the SIG node meeting where this was discussed as a result of PTO. I am wondering why the static pod author is not able to set priorityClassName on their manifest to system-cluster-critical ? we have been doing this, and i had not seen an issue with that approach.

bsalamat · 2019-08-06T22:59:21Z

I am not necessarily adverse to this. I know I was out of the SIG node meeting where this was discussed as a result of PTO. I am wondering why the static pod author is not able to set priorityClassName on their manifest to system-cluster-critical ? we have been doing this, and i had not seen an issue with that approach.

@derekwaynecarr They can set priorityClassName in their podspec, but the integer value of the priority will not be resolved by time a static pod goes through admission logic of Kubelet. Please see #80489 for more info.

dchen1107 · 2019-08-08T00:02:42Z

@derekwaynecarr We discussed this at SIG Node, and I documented what we discussed at the original issue: #80203 (comment) I believe I also recorded the meeting. Please let me know if you had a second thought on it. Otherwise, I plan to approve the pr. Thanks!

dchen1107 · 2019-08-08T00:04:27Z

/lgtm

Will wait for @derekwaynecarr before giving the final approval.

derekwaynecarr · 2019-08-08T17:19:22Z

@bsalamat thanks for the missing detail.

/approve

draveness · 2019-08-08T17:30:56Z

/retest

dchen1107 · 2019-08-08T18:07:12Z

/approve

k8s-ci-robot · 2019-08-08T18:07:51Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dchen1107, derekwaynecarr, hpandeycodeit

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~pkg/kubelet/OWNERS~~ [dchen1107,derekwaynecarr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. sig/node Categorizes an issue or PR as relevant to SIG Node. area/kubelet and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jul 23, 2019

k8s-ci-robot requested review from derekwaynecarr and tmrts July 23, 2019 21:05

draveness reviewed Jul 24, 2019

View reviewed changes

k8s-ci-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jul 24, 2019

mattjmcnaughton reviewed Jul 24, 2019

View reviewed changes

bsalamat reviewed Jul 24, 2019

View reviewed changes

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jul 24, 2019

bsalamat reviewed Jul 25, 2019

View reviewed changes

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Jul 25, 2019

bsalamat added this to the v1.16 milestone Jul 25, 2019

bsalamat reviewed Jul 27, 2019

View reviewed changes

k8s-ci-robot assigned bsalamat Jul 27, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 27, 2019

hpandeycodeit force-pushed the static_pod_priority branch from 9fdd1c3 to 9e1aa1a Compare July 29, 2019 18:38

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 29, 2019

k8s-ci-robot assigned dchen1107 Jul 29, 2019

bsalamat reviewed Jul 29, 2019

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 29, 2019

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 5, 2019

k8s-ci-robot added do-not-merge/contains-merge-commits Indicates a PR which contains merge commits. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Aug 7, 2019

changed IsCriticalPod to return true in case of static pods

c05d506

hpandeycodeit force-pushed the static_pod_priority branch from 9bea463 to c05d506 Compare August 7, 2019 22:48

k8s-ci-robot removed the do-not-merge/contains-merge-commits Indicates a PR which contains merge commits. label Aug 7, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 8, 2019

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 8, 2019

sjenning mentioned this pull request Aug 8, 2019

Bug 1739601: UPSTREAM: 80491: Changed IsCriticalPod to return true in case of static pods openshift/origin#23576

Merged

k8s-ci-robot merged commit 806ced3 into kubernetes:master Aug 8, 2019

This was referenced Aug 9, 2019

kubelet refuses to admit critical static pod #80203

Closed

feat: cleanup pod critical pod annotations feature #80342

Merged

sjenning mentioned this pull request Aug 9, 2019

Bug 1739603: UPSTREAM: 80491: Changed IsCriticalPod to return true in case of static pods [4.1] openshift/origin#23584

Merged

This was referenced Apr 8, 2020

Static control plane pods do not have priority set #89966

Closed

Fix priorityClassName typo, add numeric priority to static pods #89970

Merged

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changed IsCriticalPod to return true in case of static pods #80491

Changed IsCriticalPod to return true in case of static pods #80491

hpandeycodeit commented Jul 23, 2019 •

edited by bsalamat

Loading

hpandeycodeit commented Jul 23, 2019

hpandeycodeit commented Jul 23, 2019

draveness left a comment

mattjmcnaughton left a comment

mattjmcnaughton Jul 24, 2019

hpandeycodeit Jul 24, 2019 •

edited

Loading

mattjmcnaughton Jul 24, 2019

bsalamat left a comment

hpandeycodeit commented Jul 24, 2019

bsalamat Jul 25, 2019

hpandeycodeit commented Jul 25, 2019

bsalamat left a comment

hpandeycodeit commented Jul 29, 2019

bsalamat left a comment

derekwaynecarr commented Aug 6, 2019

bsalamat commented Aug 6, 2019

dchen1107 commented Aug 8, 2019

dchen1107 commented Aug 8, 2019

derekwaynecarr commented Aug 8, 2019

draveness commented Aug 8, 2019

dchen1107 commented Aug 8, 2019

k8s-ci-robot commented Aug 8, 2019

Changed IsCriticalPod to return true in case of static pods #80491

Changed IsCriticalPod to return true in case of static pods #80491

Conversation

hpandeycodeit commented Jul 23, 2019 • edited by bsalamat Loading

hpandeycodeit commented Jul 23, 2019

hpandeycodeit commented Jul 23, 2019

draveness left a comment

Choose a reason for hiding this comment

mattjmcnaughton left a comment

Choose a reason for hiding this comment

mattjmcnaughton Jul 24, 2019

Choose a reason for hiding this comment

hpandeycodeit Jul 24, 2019 • edited Loading

Choose a reason for hiding this comment

mattjmcnaughton Jul 24, 2019

Choose a reason for hiding this comment

bsalamat left a comment

Choose a reason for hiding this comment

hpandeycodeit commented Jul 24, 2019

bsalamat Jul 25, 2019

Choose a reason for hiding this comment

hpandeycodeit commented Jul 25, 2019

bsalamat left a comment

Choose a reason for hiding this comment

hpandeycodeit commented Jul 29, 2019

bsalamat left a comment

Choose a reason for hiding this comment

derekwaynecarr commented Aug 6, 2019

bsalamat commented Aug 6, 2019

dchen1107 commented Aug 8, 2019

dchen1107 commented Aug 8, 2019

derekwaynecarr commented Aug 8, 2019

draveness commented Aug 8, 2019

dchen1107 commented Aug 8, 2019

k8s-ci-robot commented Aug 8, 2019

hpandeycodeit commented Jul 23, 2019 •

edited by bsalamat

Loading

hpandeycodeit Jul 24, 2019 •

edited

Loading