LoadBalancerIP support for Amazon LoadBalancer

[sergey-safarov]

What would you like to be added:
Amazon cloud support addition elastic IP as loadbalancer external IP. Screenshot attached.
But I cannot use this featured as not supported for Kubernetes 1.13. I receive error LoadBalancerIP cannot be specified for AWS ELB.

yaml content

apiVersion: v1
kind: Service
metadata:
  name: proxy-dc0-0
spec:
  selector:
    statefulset.kubernetes.io/pod-name: proxy-dc0-0
  type: LoadBalancer
  loadBalancerIP: 34.211.156.102
  ports:
  - name: sip
    port: 5060
    protocol: TCP
    targetPort: 5060

Service addition

[safarov@safarov-dell kube]$ kubectl create -f sip_service3.yaml
service/proxy-dc0-0 created

Server status

[safarov@safarov-dell kube]$ kubectl describe  service proxy-dc0-0
Name:                     proxy-dc0-0
Namespace:                ippbx
Labels:                   <none>
Annotations:              <none>
Selector:                 statefulset.kubernetes.io/pod-name=proxy-dc0-0
Type:                     LoadBalancer
IP:                       10.100.113.135
IP:                       34.211.156.102
Port:                     sip  5060/TCP
TargetPort:               5060/TCP
NodePort:                 sip  30797/TCP
Endpoints:                
Session Affinity:         None
External Traffic Policy:  Cluster
Events:
  Type     Reason                      Age               From                Message
  ----     ------                      ----              ----                -------
  Normal   EnsuringLoadBalancer        8s (x2 over 13s)  service-controller  Ensuring load balancer
  Warning  CreatingLoadBalancerFailed  8s (x2 over 13s)  service-controller  Error creating load balancer (will retry): failed to ensure load balancer for service ippbx/proxy-dc0-0: LoadBalancerIP cannot be specified for AWS ELB

Why is this needed:
This feature need because many customers use firewalls and need use static IP on our server side.

/kind feature

[plumdog]

It seems that support for this did find its way into the in-tree cloud provider, but is missing from this out-of-tree provider. The original PR is here kubernetes/kubernetes#69263

@brooksgarrett it was your work originally got this support into the cloud provider, but it would seem to have got lost in the move to the separate repo. By the looks of things, the original work never made it into a non-alpha/beta release of Kubernetes. So, in a sense this is a feature, in another sense it is a nearly a regression.

I'm happy to open a PR that is shamelessly the original PR to the main kubernetes repo, but rejigged as best I can, or it might be simpler (and more honest) if @brooksgarrett were to do that, if they happen to have the time.

[brooksgarrett]

It was nearly a year from opening the PR to merge so I'm completely out of touch with the code base. I'll look when I have a moment and see if I can get it sorted.

[plumdog]

Looking some more, might LoadBalancerIP be a better place to specify this, see https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer

[izakp]

Looking for this feature as well to host our top-level domian proxy app on Kubernetes rather than a standalone server

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[JnMik]

I would need this too.
To make sure my load balancer is always using the same ip, even if I destroy / respawn it. so I don't have to always edit a CNAME to point to another generated lb aws dns name

[sergey-safarov]

/remove-lifecycle stale

[bfmiv]

This is supported in k8s 1.16.0+ via the service.beta.kubernetes.io/aws-load-balancer-eip-allocations annotation:

---
apiVersion: v1
kind: Service
metadata:
  annotations:
    # ...
    service.beta.kubernetes.io/aws-load-balancer-eip-allocations: >-
      eipalloc-0123456789abcdefg,
      eipalloc-0123456789hijklmn,
      eipalloc-0123456789opqrstu
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
  name: test-service
spec:
  selector:
    app: test-service
  externalTrafficPolicy: Local
  ports:
    # ...
  type: LoadBalancer

[zimmertr]

I'm confused, so you can specify an Elastic IP to associate with the service but not an existing Load Balancer's ARN? How does this solve the problem of reusing an existing Load Balancer?

[jmturwy]

I'm confused, so you can specify an Elastic IP to associate with the service but not an existing Load Balancer's ARN? How does this solve the problem of reusing an existing Load Balancer?

In AWS once a NLB is created you cannot change the subnets or assign an EIP. This is an aws limitation not k8

[sskmail14]

This is supported in k8s 1.16.0+ via the service.beta.kubernetes.io/aws-load-balancer-eip-allocations annotation:

---
apiVersion: v1
kind: Service
metadata:
  annotations:
    # ...
    service.beta.kubernetes.io/aws-load-balancer-eip-allocations: >-
      eipalloc-0123456789abcdefg,
      eipalloc-0123456789hijklmn,
      eipalloc-0123456789opqrstu
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
  name: test-service
spec:
  selector:
    app: test-service
  externalTrafficPolicy: Local
  ports:
    # ...
  type: LoadBalancer

@bfmiv
Sounds good.

What is the External IP assigned for the service?
If so how did you point in R53 just as a IPv4 address?

[sergey-safarov]

What is the External IP assigned for the service?

Will be used IP associated with this allocations

      eipalloc-0123456789abcdefg,
      eipalloc-0123456789hijklmn,
      eipalloc-0123456789opqrstu

If so how did you point in R53 just as a IPv4 address?

You can resolve IP address using AWS API from IP allocation.

[randomvariable]

comment from @andrewsykim : kube-proxy will inject rules to take traffic for anything with "LoadBalancerIP" and will be routed internally.
Question as to whether this is intended behaviour, because of things like TLS termination or routing.

[nckturner]

/assign

[nckturner]

@sergey-safarov does the annotation mentioned above fit your use case? Should we close this issue?

[sergey-safarov]

Hi Nicholas @nckturner
Now I not able to verify this feature.
I will do it when start work with EKS again. So for now I close issue.

Thank you very much Nicholas