Support for Standard LB in k8s v1.11 #3515

sozercan · 2018-07-19T17:58:14Z

What this PR does / why we need it:
Adds support for Standard LB for Kubernetes v1.11

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #3468

Release note:

Support for Standard Load Balancer and Public IP in Kubernetes 1.11

feiskyer · 2018-07-20T04:33:02Z

@sozercan Thanks for adding this. We may only apply the options for master nodes, as they're only used in kube-controller-manager or cloud-controller-manager.

sozercan · 2018-07-20T05:55:45Z

@feiskyer i can add it to provisionScriptParametersMaster only but azure.json is same on master and agents https://github.com/Azure/acs-engine/blob/master/parts/k8s/kubernetescustomscript.sh#L230

feiskyer · 2018-07-20T06:39:26Z

Yep, I see. It's also ok to set it in node's configs.

codecov · 2018-07-20T06:39:40Z

Codecov Report

Merging #3515 into master will decrease coverage by 0.06%.
The diff coverage is 48.27%.

@@            Coverage Diff             @@
##           master    #3515      +/-   ##
==========================================
- Coverage   55.46%   55.39%   -0.07%     
==========================================
  Files         108      108              
  Lines       16069    16098      +29     
==========================================
+ Hits         8912     8918       +6     
- Misses       6389     6413      +24     
+ Partials      768      767       -1

CecileRobertMichon · 2018-07-20T17:21:20Z

docs/clusterdefinition.md

+| serviceCidr                     | no       | IP range for Service IPs, Default is "10.0.0.0/16". This range is never routed outside of a node so does not need to lie within clusterSubnet or the VNET                                                                                                                                                                                                                                                                                                            |
+| useInstanceMetadata             | no       | Use the Azure cloudprovider instance metadata service for appropriate resource discovery operations. Default is `true`                                                                                                                                                                                                                                                                                                                                               |
+| useManagedIdentity              | no       | Includes and uses MSI identities for all interactions with the Azure Resource Manager (ARM) API. Instead of using a static service principal written to /etc/kubernetes/azure.json, Kubernetes will use a dynamic, time-limited token fetched from the MSI extension running on master and agent nodes. This support is currently alpha and requires Kubernetes v1.9.1 or newer. (boolean - default == false)                                                        |
+| loadBalancerSku                 | no       | Sku of Load Balancer and Public IP. Candidate values are: `basic` and `standard`. If not set, it will be default to basic. Requires Kubernetes 1.11 or newer.                                                                                                                                                                                                                                                                                                            |


Only standard requires 1.11 correct? "standard requires Kubernetes 1.11 or newer" might be more clear

CecileRobertMichon · 2018-07-20T17:23:45Z

docs/clusterdefinition.md

+| useInstanceMetadata             | no       | Use the Azure cloudprovider instance metadata service for appropriate resource discovery operations. Default is `true`                                                                                                                                                                                                                                                                                                                                               |
+| useManagedIdentity              | no       | Includes and uses MSI identities for all interactions with the Azure Resource Manager (ARM) API. Instead of using a static service principal written to /etc/kubernetes/azure.json, Kubernetes will use a dynamic, time-limited token fetched from the MSI extension running on master and agent nodes. This support is currently alpha and requires Kubernetes v1.9.1 or newer. (boolean - default == false)                                                        |
+| loadBalancerSku                 | no       | Sku of Load Balancer and Public IP. Candidate values are: `basic` and `standard`. If not set, it will be default to basic. Requires Kubernetes 1.11 or newer.                                                                                                                                                                                                                                                                                                            |
+| excludeMasterFromStandardLB     | no       | Excludes master nodes from standard load balancer. Default is `true`. Requires Kubernetes 1.11 or newer.                                                                                                                                                                                                                                                                                                                                                             |


Same thing here, which part requires 1.11? If the default is always true then what happens if the user is on k8s 1.8?

I believe it'll just be ignored by k8s v1.10 or earlier (since anything but v1.11 doesn't check for those values). @feiskyer can you confirm?

CecileRobertMichon · 2018-07-20T17:24:27Z

parts/k8s/kuberneteswindowssetup.ps1

@@ -65,6 +65,8 @@ $global:KubeDnsSearchPath = "svc.cluster.local"

 $global:UseManagedIdentityExtension = "{{WrapAsVariable "useManagedIdentityExtension"}}"
 $global:UseInstanceMetadata = "{{WrapAsVariable "useInstanceMetadata"}}"
+$global:LoadBalancerSku = "{{WrapAsVariable "loadBalancerSku"}}"


@PatrickLang could you please review this file?

Sure - the changes here look good to me. I don't see any Windows-specific things changing. The only questions from me are whether this needs an azure-cni update and if the cloud provider already supports these in K8s 1.9 - 1.11

This code wouldn't need to change, just checking that those are updated if required when this is merged

@PatrickLang k8s v1.11 is required for this, tested this on windows pool and it successfully created standard lb and public ip.

Thanks @PatrickLang

CecileRobertMichon · 2018-07-20T17:46:10Z

docs/clusterdefinition.md

+| useInstanceMetadata             | no       | Use the Azure cloudprovider instance metadata service for appropriate resource discovery operations. Default is `true`                                                                                                                                                                                                                                                                                                                                               |
+| useManagedIdentity              | no       | Includes and uses MSI identities for all interactions with the Azure Resource Manager (ARM) API. Instead of using a static service principal written to /etc/kubernetes/azure.json, Kubernetes will use a dynamic, time-limited token fetched from the MSI extension running on master and agent nodes. This support is currently alpha and requires Kubernetes v1.9.1 or newer. (boolean - default == false)                                                        |
+| loadBalancerSku                 | no       | Sku of Load Balancer and Public IP. Candidate values are: `basic` and `standard`. If not set, defaults to `basic`. `standard` requires Kubernetes 1.11 or newer.                                                                                                                                                                                                                                                                                                     |
+| excludeMasterFromStandardLB     | no       | Excludes master nodes from standard load balancer. Requires `loadBalancerSku` to be set to `standard` and Kubernetes v1.11 or newer. Default is `true`.                                                                                                                                                                                                                                                                                                              |


I still think that this default behavior is counter intuitive since we are always defaulting to true even though that is only supported for k8s 1.11 with standard LB.... which means that we are defaulting to a non-supported option

should we flip around the setting to includeMasterInStandardLB? @feiskyer thoughts?

Could we set excludeMasterFromStandardLB only when it is configured explicitly by the user? The default true is recommended.

includeMasterInStandardLB is actually same with excludeMasterFromStandardLB, just with a different default value. Since it's a k8s configure option, it's better to keep it as is.

Could the default behavior be: if loadBalancerSku == standard => true, Else => false

PatrickLang

Windows change looks good to me. The tests in place do cover ingress to a Windows service so be sure to update the apimodel to test each mode before merge

PatrickLang · 2018-07-20T21:56:46Z

parts/k8s/kuberneteswindowssetup.ps1

@@ -65,6 +65,8 @@ $global:KubeDnsSearchPath = "svc.cluster.local"

 $global:UseManagedIdentityExtension = "{{WrapAsVariable "useManagedIdentityExtension"}}"
 $global:UseInstanceMetadata = "{{WrapAsVariable "useInstanceMetadata"}}"
+$global:LoadBalancerSku = "{{WrapAsVariable "loadBalancerSku"}}"


Sure - the changes here look good to me. I don't see any Windows-specific things changing. The only questions from me are whether this needs an azure-cni update and if the cloud provider already supports these in K8s 1.9 - 1.11

PatrickLang · 2018-07-20T21:57:12Z

parts/k8s/kuberneteswindowssetup.ps1

@@ -65,6 +65,8 @@ $global:KubeDnsSearchPath = "svc.cluster.local"

 $global:UseManagedIdentityExtension = "{{WrapAsVariable "useManagedIdentityExtension"}}"
 $global:UseInstanceMetadata = "{{WrapAsVariable "useInstanceMetadata"}}"
+$global:LoadBalancerSku = "{{WrapAsVariable "loadBalancerSku"}}"


This code wouldn't need to change, just checking that those are updated if required when this is merged

PatrickLang · 2018-07-20T22:00:36Z

I looked at the section @CecileRobertMichon recommended, but didn't review the rest

feiskyer · 2018-07-24T23:44:49Z

cc @khenidak

jackfrancis · 2018-07-25T19:32:02Z

code lgtm, pending rebase and E2E

sozercan · 2018-07-25T19:44:33Z

@jackfrancis rebased

khenidak · 2018-07-25T21:34:24Z

The only comment i have is. When zoned are enabled we will have to validate that this sku is set by default.

cc @ritazh @kkmsft

ritazh · 2018-07-25T23:00:20Z

@khenidak yup

jackfrancis · 2018-07-30T23:02:20Z

pkg/acsengine/defaults.go

+		}
+
+		if a.OrchestratorProfile.KubernetesConfig.ExcludeMasterFromStandardLB == nil {
+			a.OrchestratorProfile.KubernetesConfig.ExcludeMasterFromStandardLB = helpers.PointerToBool(api.DefaultExcludeMasterFromStandardLB)


As @CecileRobertMichon suggested, let's convert the default value assignment of ExcludeMasterFromStandardLB to a function. The function should look like this:

is this a 1.11 or greater cluster that is using "Standard" LB?

if so, set to true (based on comments from @feiskyer)

if not, false

jackfrancis · 2018-08-02T23:18:54Z

Our E2E tests tell us that ILB scenarios break w/ standard LB-enabled clusters. @feiskyer do you have an upstream bug that this may be related to?

ritazh · 2018-08-03T17:31:05Z

This is a prereq for #3453.

feiskyer · 2018-08-05T13:05:20Z

Our E2E tests tell us that ILB scenarios break w/ standard LB-enabled clusters. @feiskyer do you have an upstream bug that this may be related to?

Which region are you using? Could you verify whether it's a Standard LB issue (e.g. without k8s and enabling floatingIP)? I have met that before and standard LB with floatingIP enabled may not work on some regions.

sozercan · 2018-08-06T15:59:41Z

@feiskyer acs-engine CI failed on southcentralus, westcentralus, southeastasia, westeurope (it failed on all 4 runs)

This is the test that fails which tests working ILB:
https://github.com/Azure/acs-engine/blob/master/test/e2e/kubernetes/kubernetes_test.go#L503
https://github.com/Azure/acs-engine/blob/master/test/e2e/kubernetes/workloads/ingress-nginx-ilb.yaml

feiskyer · 2018-08-07T03:30:09Z

@sozercan Thanks. So it's a standard ILB issue, which disables external connectivity by default.

jackfrancis · 2018-08-07T16:35:48Z

@sozercan @feiskyer What are the next steps here?

khenidak · 2018-08-08T16:41:52Z

Hold, we are in talks with Networking team for clarity on what we can do. Should resolve this week

ritazh · 2018-08-13T16:38:32Z

@khenidak @jackfrancis I've added the elb service as a workaround. PTAL

sozercan · 2018-08-14T01:33:29Z

parts/k8s/addons/kubernetesmasteraddons-elb-svc.yaml

+  - port: 8765
+    targetPort: 9376
+  selector:
+    app: random


are you keeping this as random?

khenidak · 2018-08-14T15:25:05Z

docs/clusterdefinition.md

@@ -53,6 +53,7 @@ Here are the valid values for the orchestrator types:
 | gcLowThreshold                  | no       | Sets the --image-gc-low-threshold value on the kublet configuration. Default is 80. [See kubelet Garbage Collection](https://kubernetes.io/docs/concepts/cluster-administration/kubelet-garbage-collection/)                                                                                                                                                                                                  |
 | kubeletConfig                   | no       | Configure various runtime configuration for kubelet. See `kubeletConfig` [below](#feat-kubelet-config)                                                                                                                                                                                                                                                                                                        |
 | kubernetesImageBase             | no       | Specifies the base URL (everything preceding the actual image filename) of the kubernetes hyperkube image to use for cluster deployment, e.g., `k8s.gcr.io/`                                                                                                                                                                                                                                     |
+| loadBalancerSku                 | no       | Sku of Load Balancer and Public IP. Candidate values are: `basic` and `standard`. If not set, it will be default to basic. Requires Kubernetes 1.11 or newer.                                                                                                                                                                                                                                                                                                        |


Please add documentation for LB outbound rule workaround + what is expected in v1.12

khenidak

missing documentation for the outbound rule workaround

ritazh · 2018-08-14T23:55:54Z

@khenidak @jackfrancis @sozercan doc added, rebased, ILB e2e test on jenkins passed, PTAL

jackfrancis · 2018-08-14T23:58:03Z

parts/k8s/kubernetesmastervars.t

@@ -67,6 +67,8 @@
 {{end}}
    "useManagedIdentityExtension": "{{ UseManagedIdentity }}",
    "useInstanceMetadata": "{{ UseInstanceMetadata }}",
+    "loadBalancerSku": "{{ LoadBalancerSku }}",
+    "excludeMasterFromStandardLB": "{{ ExcludeMasterFromStandardLB }}",


you can get rid of this variable assignment...

These two are variables, not parameters.

jackfrancis · 2018-08-14T23:58:46Z

parts/k8s/kubernetesmastervars.t

@@ -132,7 +134,7 @@
    "customSearchDomainsScript": "{{GetKubernetesB64CustomSearchDomainsScript}}",
    "sshdConfig": "{{GetB64sshdConfig}}",
 {{if not IsOpenShift}}
-    "provisionScriptParametersCommon": "[concat('ADMINUSER=',parameters('linuxAdminUsername'),' ETCD_DOWNLOAD_URL=',parameters('etcdDownloadURLBase'),' ETCD_VERSION=',parameters('etcdVersion'),' DOCKER_ENGINE_VERSION=',parameters('dockerEngineVersion'),' DOCKER_REPO=',parameters('dockerEngineDownloadRepo'),' TENANT_ID=',variables('tenantID'),' HYPERKUBE_URL=',parameters('kubernetesHyperkubeSpec'),' APISERVER_PUBLIC_KEY=',parameters('apiserverCertificate'),' SUBSCRIPTION_ID=',variables('subscriptionId'),' RESOURCE_GROUP=',variables('resourceGroup'),' LOCATION=',variables('location'),' VM_TYPE=',variables('vmType'),' SUBNET=',variables('subnetName'),' NETWORK_SECURITY_GROUP=',variables('nsgName'),' VIRTUAL_NETWORK=',variables('virtualNetworkName'),' VIRTUAL_NETWORK_RESOURCE_GROUP=',variables('virtualNetworkResourceGroupName'),' ROUTE_TABLE=',variables('routeTableName'),' PRIMARY_AVAILABILITY_SET=',variables('primaryAvailabilitySetName'),' PRIMARY_SCALE_SET=',variables('primaryScaleSetName'),' SERVICE_PRINCIPAL_CLIENT_ID=',variables('servicePrincipalClientId'),' SERVICE_PRINCIPAL_CLIENT_SECRET=',variables('singleQuote'),variables('servicePrincipalClientSecret'),variables('singleQuote'),' KUBELET_PRIVATE_KEY=',parameters('clientPrivateKey'),' TARGET_ENVIRONMENT=',parameters('targetEnvironment'),' NETWORK_PLUGIN=',parameters('networkPlugin'),' VNET_CNI_PLUGINS_URL=',parameters('vnetCniLinuxPluginsURL'),' CNI_PLUGINS_URL=',parameters('cniPluginsURL'),' CLOUDPROVIDER_BACKOFF=',parameters('cloudproviderConfig').cloudProviderBackoff,' CLOUDPROVIDER_BACKOFF_RETRIES=',parameters('cloudproviderConfig').cloudProviderBackoffRetries,' CLOUDPROVIDER_BACKOFF_EXPONENT=',parameters('cloudproviderConfig').cloudProviderBackoffExponent,' CLOUDPROVIDER_BACKOFF_DURATION=',parameters('cloudproviderConfig').cloudProviderBackoffDuration,' CLOUDPROVIDER_BACKOFF_JITTER=',parameters('cloudproviderConfig').cloudProviderBackoffJitter,' CLOUDPROVIDER_RATELIMIT=',parameters('cloudproviderConfig').cloudProviderRatelimit,' CLOUDPROVIDER_RATELIMIT_QPS=',parameters('cloudproviderConfig').cloudProviderRatelimitQPS,' CLOUDPROVIDER_RATELIMIT_BUCKET=',parameters('cloudproviderConfig').cloudProviderRatelimitBucket,' USE_MANAGED_IDENTITY_EXTENSION=',variables('useManagedIdentityExtension'),' USE_INSTANCE_METADATA=',variables('useInstanceMetadata'),' CONTAINER_RUNTIME=',parameters('containerRuntime'),' CONTAINERD_DOWNLOAD_URL_BASE=',parameters('containerdDownloadURLBase'),' POD_INFRA_CONTAINER_SPEC=',parameters('kubernetesPodInfraContainerSpec'),' KMS_PROVIDER_VAULT_NAME=',variables('clusterKeyVaultName'))]",
+    "provisionScriptParametersCommon": "[concat('ADMINUSER=',parameters('linuxAdminUsername'),' ETCD_DOWNLOAD_URL=',parameters('etcdDownloadURLBase'),' ETCD_VERSION=',parameters('etcdVersion'),' DOCKER_ENGINE_VERSION=',parameters('dockerEngineVersion'),' DOCKER_REPO=',parameters('dockerEngineDownloadRepo'),' TENANT_ID=',variables('tenantID'),' HYPERKUBE_URL=',parameters('kubernetesHyperkubeSpec'),' APISERVER_PUBLIC_KEY=',parameters('apiserverCertificate'),' SUBSCRIPTION_ID=',variables('subscriptionId'),' RESOURCE_GROUP=',variables('resourceGroup'),' LOCATION=',variables('location'),' VM_TYPE=',variables('vmType'),' SUBNET=',variables('subnetName'),' NETWORK_SECURITY_GROUP=',variables('nsgName'),' VIRTUAL_NETWORK=',variables('virtualNetworkName'),' VIRTUAL_NETWORK_RESOURCE_GROUP=',variables('virtualNetworkResourceGroupName'),' ROUTE_TABLE=',variables('routeTableName'),' PRIMARY_AVAILABILITY_SET=',variables('primaryAvailabilitySetName'),' PRIMARY_SCALE_SET=',variables('primaryScaleSetName'),' SERVICE_PRINCIPAL_CLIENT_ID=',variables('servicePrincipalClientId'),' SERVICE_PRINCIPAL_CLIENT_SECRET=',variables('singleQuote'),variables('servicePrincipalClientSecret'),variables('singleQuote'),' KUBELET_PRIVATE_KEY=',parameters('clientPrivateKey'),' TARGET_ENVIRONMENT=',parameters('targetEnvironment'),' NETWORK_PLUGIN=',parameters('networkPlugin'),' VNET_CNI_PLUGINS_URL=',parameters('vnetCniLinuxPluginsURL'),' CNI_PLUGINS_URL=',parameters('cniPluginsURL'),' CLOUDPROVIDER_BACKOFF=',parameters('cloudproviderConfig').cloudProviderBackoff,' CLOUDPROVIDER_BACKOFF_RETRIES=',parameters('cloudproviderConfig').cloudProviderBackoffRetries,' CLOUDPROVIDER_BACKOFF_EXPONENT=',parameters('cloudproviderConfig').cloudProviderBackoffExponent,' CLOUDPROVIDER_BACKOFF_DURATION=',parameters('cloudproviderConfig').cloudProviderBackoffDuration,' CLOUDPROVIDER_BACKOFF_JITTER=',parameters('cloudproviderConfig').cloudProviderBackoffJitter,' CLOUDPROVIDER_RATELIMIT=',parameters('cloudproviderConfig').cloudProviderRatelimit,' CLOUDPROVIDER_RATELIMIT_QPS=',parameters('cloudproviderConfig').cloudProviderRatelimitQPS,' CLOUDPROVIDER_RATELIMIT_BUCKET=',parameters('cloudproviderConfig').cloudProviderRatelimitBucket,' USE_MANAGED_IDENTITY_EXTENSION=',variables('useManagedIdentityExtension'),' USE_INSTANCE_METADATA=',variables('useInstanceMetadata'),' LOAD_BALANCER_SKU=',variables('loadBalancerSku'),' EXCLUDE_MASTER_FROM_STANDARD_LB=',variables('excludeMasterFromStandardLB'),' CONTAINER_RUNTIME=',parameters('containerRuntime'),' CONTAINERD_DOWNLOAD_URL_BASE=',parameters('containerdDownloadURLBase'),' POD_INFRA_CONTAINER_SPEC=',parameters('kubernetesPodInfraContainerSpec'),' KMS_PROVIDER_VAULT_NAME=',variables('clusterKeyVaultName'))]",


... and change to parameters('excludeMasterFromStandardLB')

Same here, these are variables, not parameters.

jackfrancis · 2018-08-14T23:59:13Z

parts/k8s/kuberneteswindowssetup.ps1

@@ -65,6 +65,8 @@ $global:KubeDnsSearchPath = "svc.cluster.local"

 $global:UseManagedIdentityExtension = "{{WrapAsVariable "useManagedIdentityExtension"}}"
 $global:UseInstanceMetadata = "{{WrapAsVariable "useInstanceMetadata"}}"
+$global:LoadBalancerSku = "{{WrapAsVariable "loadBalancerSku"}}"
+$global:ExcludeMasterFromStandardLB = "{{WrapAsVariable "excludeMasterFromStandardLB"}}"


... and change to WrapAsParameter "excludeMasterFromStandardLB"

jackfrancis · 2018-08-15T00:00:18Z

added a fun comment about superfluous variables and then lgtm!

khenidak · 2018-08-15T00:01:36Z

/lgtm

acs-bot · 2018-08-15T00:01:39Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: khenidak, sozercan
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: jackfrancis

If they are not already assigned, you can assign the PR to them by writing /assign @jackfrancis in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

acs-bot · 2018-08-15T00:01:44Z

@khenidak: changing LGTM is restricted to assignees, and only Azure/acs-engine repo collaborators may be assigned issues.

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jackfrancis · 2018-08-15T00:20:14Z

@ritazh yeah, nevermind, this works fine for now, thanks!

ghost assigned sozercan Jul 19, 2018

ghost added the in progress label Jul 19, 2018

sozercan force-pushed the loadBalancerSku branch 2 times, most recently from 63e9936 to ee14ae9 Compare July 20, 2018 03:46

sozercan changed the title ~~[WIP] Support for Standard LB in k8s v1.11~~ Support for Standard LB in k8s v1.11 Jul 20, 2018

sozercan requested a review from CecileRobertMichon July 20, 2018 17:15

sozercan added the orchestrator/k8s label Jul 20, 2018

CecileRobertMichon reviewed Jul 20, 2018

View reviewed changes

sozercan force-pushed the loadBalancerSku branch from d4d1f7d to 1ee0c6a Compare July 20, 2018 17:41

CecileRobertMichon reviewed Jul 20, 2018

View reviewed changes

PatrickLang approved these changes Jul 20, 2018

View reviewed changes

sozercan force-pushed the loadBalancerSku branch from 1ee0c6a to 936c319 Compare July 25, 2018 19:44

acs-bot added the size/L label Jul 25, 2018

jackfrancis mentioned this pull request Jul 27, 2018

Kubernetes: add —no-negcache to dnsmasq config #3564

Closed

3 tasks

jackfrancis reviewed Jul 30, 2018

View reviewed changes

sozercan force-pushed the loadBalancerSku branch from 936c319 to 80d1fdb Compare July 31, 2018 03:11

ghost assigned ritazh Aug 10, 2018

ritazh force-pushed the loadBalancerSku branch from 9ccbee1 to dcb3d02 Compare August 10, 2018 23:10

sozercan commented Aug 14, 2018

View reviewed changes

ritazh force-pushed the loadBalancerSku branch from 2a9995e to 9a67497 Compare August 14, 2018 06:42

khenidak reviewed Aug 14, 2018

View reviewed changes

khenidak suggested changes Aug 14, 2018

View reviewed changes

ritazh force-pushed the loadBalancerSku branch 3 times, most recently from f2806f1 to ad88ed6 Compare August 14, 2018 22:53

add loadBalancerSku and excludeMasterFromStandardLB

f508919

ritazh force-pushed the loadBalancerSku branch from ad88ed6 to f508919 Compare August 14, 2018 22:57

jackfrancis reviewed Aug 14, 2018

View reviewed changes

jackfrancis merged commit 1ade2f3 into Azure:master Aug 15, 2018

ghost removed the in progress label Aug 15, 2018

sozercan deleted the loadBalancerSku branch August 15, 2018 00:27

Support for Standard LB in k8s v1.11 #3515

Support for Standard LB in k8s v1.11 #3515

Conversation

sozercan commented Jul 19, 2018 • edited Loading

feiskyer commented Jul 20, 2018 • edited Loading

sozercan commented Jul 20, 2018

feiskyer commented Jul 20, 2018

codecov bot commented Jul 20, 2018 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PatrickLang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PatrickLang commented Jul 20, 2018

feiskyer commented Jul 24, 2018

jackfrancis commented Jul 25, 2018

sozercan commented Jul 25, 2018

khenidak commented Jul 25, 2018

ritazh commented Jul 25, 2018

Choose a reason for hiding this comment

jackfrancis commented Aug 2, 2018

ritazh commented Aug 3, 2018

feiskyer commented Aug 5, 2018

sozercan commented Aug 6, 2018

feiskyer commented Aug 7, 2018

jackfrancis commented Aug 7, 2018

khenidak commented Aug 8, 2018

ritazh commented Aug 13, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

khenidak left a comment

Choose a reason for hiding this comment

ritazh commented Aug 14, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jackfrancis commented Aug 15, 2018

khenidak commented Aug 15, 2018

acs-bot commented Aug 15, 2018

acs-bot commented Aug 15, 2018

jackfrancis commented Aug 15, 2018

sozercan commented Jul 19, 2018 •

edited

Loading

feiskyer commented Jul 20, 2018 •

edited

Loading

codecov bot commented Jul 20, 2018 •

edited

Loading