warning in kube-proxy logs

[LuckySB]

deploy from current master,
kube_proxy_mode: ipvs
i see warnings in kube-proxy log

How dangerous are these warnings?
Given that changes are made in the role of win_nodes, and apply buy pr
#3708
#3714

kubectl logs kube-proxy-2qqj5

W0112 22:47:44.651776       1 node.go:103] Failed to retrieve node info: nodes "${node_name}" not found
I0112 22:47:44.651808       1 server_others.go:189] Using ipvs Proxier.
W0112 22:47:44.651922       1 proxier.go:350] invalid nodeIP, initializing kube-proxy with 127.0.0.1 as nodeIP

from ds manifest:

      containers:
      - command:
        - /usr/local/bin/kube-proxy
        - --config=/var/lib/kube-proxy/config.conf
        - --hostname-override=$(NODE_NAME)
        - --hostname-override=${NODE_NAME}
        env:
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        image: gcr.io/google-containers/kube-proxy:v1.13.1

ps ax | grep proxy

/usr/local/bin/kube-proxy --config=/var/lib/kube-proxy/config.conf --hostname-override=master-1.southbridge.io --hostname-override=${NODE_NAME}```

[LuckySB]

kubernetes/kubeadm#857

kubernetes/kubernetes#71283

[arianvp]

This basically breaks externalTrafficPolicy: Local. The patch that was applied has a syntax error . It uses ${NODE_NAME} which is not valid syntax instead of $(NODE_NAME). kube-proxy then silently falls back to 127.0.0.1 as the nodeIp because the literal string ${NODE_NAME} is not valid syntax. which makes it drop all the traffic that does not come from 127.0.0.1. This will then cause your cluster to not accept any external traffic on type: LoadBalancer and type: NodePort services. This was quite an interesting outage to track down. Just leaving this note here if anybody else ends up on this issue through Google.

Applying https://github.com/kubernetes-sigs/kubespray/pull/4185/files will not magically fix your cluster, as the patch isn't reverted. Instead you should do this:

kubectl edit -n kube-system daemonset/kube-proxy

and replace

    spec:
      containers:
      - command:
        - /usr/local/bin/kube-proxy
        - --config=/var/lib/kube-proxy/config.conf
        - --hostname-override=$(NODE_NAME)
        - --hostname-override=${NODE_NAME}

With:

    spec:
      containers:
      - command:
        - /usr/local/bin/kube-proxy
        - --config=/var/lib/kube-proxy/config.conf
        - --hostname-override=$(NODE_NAME)

Your cluster should now be reachable by the outside world.

[bluskool]

Thanks so much @arianvp! We ran into an issue when using custom DHCP option sets and NLBs in our EKS clusters. We attempted to patch the kube-proxy daemonset as was suggested here, but that didn't work because of the syntax error you mentioned above. Using $(NODE_NAME) instead fixed it.