Critical pods should not be limited to kube-system namespace

[bsalamat]

When we used annotations to mark critical pods, we considered pods with such an annotation critical only when they were created in "kube-system" namespace. Going to the new era of using priority to indicate critical pods, we would like to remove the restriction of "kube-system" namespace and consider pods with critical priority as critical no matter what namespace they are in.

ref/ #60519 (comment)

/kind bug
/sig scheduling
/assign @ravisantoshgudimetla
cc/ @liggitt

[ravisantoshgudimetla]

While I am fine with the approach, it seems the only way to ensure that those critical pods are not coming from some random namespace is whitelisting them using priority admission controller or using quota. Atleast previously we have one more level of security that critical pod has to be created in kube-system namespace.

[liggitt]

previously we have one more level of security that critical pod has to be created in kube-system namespace

The requirement to limit the critical pod annotation meaning to the kube-system namespace was a workaround for the lack of access control and quota around the annotation. We actively work to prevent special handling of the kube-system namespace wherever possible.

[liggitt]

the only way to ensure that those critical pods are not coming from some random namespace is whitelisting them using priority admission controller or using quota

Yes, which is why those aspects are essential to a functional priority system that works for a variety of cluster configurations

[derekwaynecarr]

Agree with Ravi and Jordan on what is needed for final solution.

[bsalamat]

There is no doubt that we need to have quota for priority. It is needed not only for critical priorities, but also for any other high priority classes that users may define. That effort is going forward and we will have it soon.

[k82cn]

@ravisantoshgudimetla , I think we also need to update Priority admission controller for that.

root@wgl-ubuntu:/home/k82cn/go_ws/src/k8s.io/kubernetes# kc create -f pod.yaml
Error from server (Forbidden): error when creating "pod.yaml": pods "rss-site" is forbidden: pods with system-cluster-critical priorityClass is not permitted in default namespace

root@wgl-ubuntu:/home/k82cn/go_ws/src/k8s.io/kubernetes# kc version
Client Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.0-alpha.1.439+535426bf736a31-dirty", GitCommit:"535426bf736a31edeb76cf64f3b6d90e6996e49a", GitTreeState:"dirty", BuildDate:"2018-05-01T07:14:55Z", GoVersion:"go1.10.1", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.0-alpha.0.1200+323e1375b3b15a", GitCommit:"323e1375b3b15a4144d46d41155ca402a2fcef63", GitTreeState:"clean", BuildDate:"2018-09-27T06:52:20Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}

root@wgl-ubuntu:/home/k82cn/go_ws/src/k8s.io/kubernetes# cat pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: rss-site
  labels:
    app: web
spec:
  priorityClassName: system-cluster-critical
  containers:
    - name: front-end
      image: nginx
      ports:
        - containerPort: 80

[liggitt]

see later PR re-restricting this until the quota controls graduate to at least beta state: #65593

[k82cn]

Got that, thanks :) That makes sense to me.

[llarsson]

If this is closed and there even was a merged pull request for this type of functionality, what is the current status? And is the documentation in sync with current status?

[nevalla]

It seems that kube-system namespace is still hard coded in Priority plugin, so I guess the current status is that system critical pods are still limited to kube-system namespace. Based on that I think this issue should not be closed.

[tlvenn]

@bsalamat can we reopen this issue please because as pointed out by @nevalla, the issue still exists. Thanks.

[zimmertr]

Waiting for this to be available myself. No reason my Node Exporter dataset has to run in the kube-system namespace instead of my Grafana namespace.

[vllry]

Confirmed code + behavior.
/open

[vllry]

/reopen

[k8s-ci-robot]

@vllry: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[nesl247]

If this is closed and there even was a merged pull request for this type of functionality, what is the current status? And is the documentation in sync with current status?

I too am confused based on that documentation. Can we get an update here with an answer and fix the documentation if it's wrong?

[liggitt]

This is resolved in 1.17

See #76310