From f6baac6c677e67698b01d9787c467890ff1f8087 Mon Sep 17 00:00:00 2001 From: caroldelwing Date: Mon, 17 Jun 2024 10:17:17 -0400 Subject: [PATCH] docs: refactor and update aws cluster autoscaler pack page (#3022) * docs: refactor and update aws cluster autoscaler pack page * docs: small fix * docs: more fixes * docs: apply suggestions from review --- .../integrations/aws-autoscaler.md | 1065 +++++++++++------ 1 file changed, 718 insertions(+), 347 deletions(-) diff --git a/docs/docs-content/integrations/aws-autoscaler.md b/docs/docs-content/integrations/aws-autoscaler.md index e129f89c91..b5ed8d4df4 100644 --- a/docs/docs-content/integrations/aws-autoscaler.md +++ b/docs/docs-content/integrations/aws-autoscaler.md @@ -7,13 +7,16 @@ type: "integration" category: ["system app", "amd64"] sidebar_class_name: "hide-from-sidebar" logoUrl: "https://registry.dev.spectrocloud.com/v1/aws-cluster-autoscaler/blobs/sha256:f86813591b3b63b3afcf0a604a7c8c715660448585e89174908f3c6a421ad8d8?type=image.webp" -tags: ["packs", "aws-cluster-autoscaler", "system app", "network"] +tags: ["packs", "aws-cluster-autoscaler", "system app", "eks"] --- -Palette supports autoscaling for AWS EKS clusters by using the AWS Cluster Autoscaler pack. The Cluster Autoscaler -dynamically scales cluster resources. It monitors the workload and provisions or shuts down cluster nodes to maximize -the cluster's performance and make it more resilient to failures. It resizes the Kubernetes cluster in the following two -conditions: +Palette supports auto scaling for Amazon Elastic Kubernetes Service (EKS) host clusters through the AWS Cluster +Autoscaler pack. + +The Cluster Autoscaler pack monitors the cluster workload and utilizes +[Amazon EC2 Auto Scaling Groups](https://docs.aws.amazon.com/autoscaling/ec2/userguide/what-is-amazon-ec2-auto-scaling.html) +to dynamically provision or shut down nodes, maximizing the cluster's performance and making it more resilient to +failures. The Kubernetes clusters are resized under the following conditions: - Scale-up: The Cluster Autoscaler triggers a scale-up operation if insufficient cluster resources lead to multiple pod failures. The pods become eligible for scheduling on the new nodes. The Cluster Autoscaler checks for pod failures @@ -26,394 +29,757 @@ conditions: threshold based on CPU and memory utilization. In scenarios where the node is underutilized, the Cluster Autoscaler migrates the pods from underutilized nodes to other available nodes and then shuts down the underutilized nodes. -Cluster Autoscaler pack is deployed as a -[_Deployment_](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) in your cluster and utilizes -[Amazon EC2 Auto Scaling Groups](https://docs.aws.amazon.com/autoscaling/ec2/userguide/what-is-amazon-ec2-auto-scaling.html) -to manage node groups. - ## Versions Supported - + -## Prerequisites +### Prerequisites -- Kubernetes 1.24.x or higher. +- An EKS host cluster with Kubernetes 1.24.x or higher. - Permission to create an IAM policy in the AWS account you use with Palette. -- IAM policy - A - [Full Cluster Autoscaler Features](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md#full-cluster-autoscaler-features-policy-recommended) - IAM policy must be attached to the EKS cluster's node group. The policy must allow the Cluster Autoscaler to scale the - cluster's node groups. - - There are two ways to achieve this prerequisite. You can define the policy as a _customer-managed_ policy in the AWS - account and use its Amazon Resource Name (ARN) in the cluster profile. Alternatively, you can attach the IAM policy as - an _inline_ policy to the node group if you have already deployed your cluster. Refer to the [Usage](#usage) section - below to learn more. - -- Updated Kubernetes layer manifest - The Kubernetes pack's manifest should be updated with the newly created IAM policy - ARN. The YAML code block below displays the `managedMachinePool.roleAdditionalPolicies` section to update in the - Kubernetes pack's manifest. Refer to the [Usage](#usage) section below for more details with an example. - -
- - ```yaml - managedMachinePool: - #roleName: {{ name of the self-managed role | format "${string}" }} - ## A list of additional policies to attach to the node group role - roleAdditionalPolicies: - - {{ arn for the policy1 | format "${string}" }} - ``` - -
- -## Usage - -Cluster Autoscaler helps improve your cluster's performance and makes your cluster more resilient to failures. It -automatically adjusts the number of nodes in your cluster based on the current workload. In other words, Cluster -Autoscaler monitors the resource utilization, such as CPU and memory, and the number of pods active in your cluster and -scales the cluster when either of these events occurs: - -
- -- Multiple pods fail due to resource contention. In this case, the Cluster Autoscaler will provision more nodes. - -- Nodes are underutilized for a specific period. In this case, the Cluster Autoscaler will reschedule the pods onto - other nodes and shut down the underutilized node. -
- -### Deploy Cluster Autoscaler - -To deploy the Cluster Autoscaler pack, you must first define an IAM policy in the AWS account associated with Palette. - -Next, update the cluster profile to specify the IAM policy ARN in the Kubernetes pack's manifest. Palette will attach -that IAM policy to your cluster's node group during deployment. Note that Palette automatically creates two IAM roles in -the AWS account when you deploy an EKS cluster. One role is for the cluster, and another for the cluster's node group. -The cluster's IAM role name will have the following naming convention, `[your-cluster-name]-iam-service-role`, and the -node group's IAM role name will follow the `ng-role_worker-pool-[random-string]` naming convention. - -The following steps provide detailed instructions for deploying the Cluster Autoscaler pack. - -
- -1. Define the new IAM policy using the policy outlined below, and give it a name, for example, - _PaletteEKSClusterAutoscaler_. -
- -```json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": [ - "autoscaling:DescribeAutoScalingGroups", - "autoscaling:DescribeAutoScalingInstances", - "autoscaling:DescribeLaunchConfigurations", - "autoscaling:DescribeScalingActivities", - "autoscaling:DescribeTags", - "ec2:DescribeInstanceTypes", - "ec2:DescribeLaunchTemplateVersions" - ], - "Resource": ["*"] - }, - { - "Effect": "Allow", - "Action": [ - "autoscaling:SetDesiredCapacity", - "autoscaling:TerminateInstanceInAutoScalingGroup", - "ec2:DescribeImages", - "ec2:GetInstanceTypesFromInstanceRequirements", - "eks:DescribeNodegroup" - ], - "Resource": ["*"] - } - ] -} -``` +:::warning -2. Copy the IAM policy ARN to the clipboard for the next step. For example, the policy ARN will be similar to - `arn:aws:iam::650628870702:policy/PaletteEKSClusterAutoscaler`. - -3. In your cluster profile, and update the `managedMachinePool.roleAdditionalPolicies` section in the Kubernetes pack's - manifest with the newly created IAM policy ARN. The snapshot below displays the specific section to update with the - policy ARN. - -![A snapshot displaying the ARN added to the Kubernetes pack's manifest.](/integrations_aws-cluster-autoscaler_k8s-manifest.webp) - -For example, the code block below displays the updated `managedMachinePool.roleAdditionalPolicies` section with a sample -policy ARN, `"arn:aws:iam::650628870702:policy/PaletteEKSClusterAutoscaler"`. Before you use the following code block, -replace the ARN below with yours. - -
- -```yaml -managedMachinePool: - # roleName: {{ name of the self-managed role | format "${string}" }} - # A list of additional policies to attach to the node group role - roleAdditionalPolicies: - - "arn:aws:iam::650628870702:policy/PaletteEKSClusterAutoscaler" -``` - -
- -:::info - -If you do not want to update the Kubernetes pack's manifest, you can add an _inline_ IAM policy to the cluster's node -group post deployment. Refer to this -[AWS guide](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html#add-policies-console) -on how to embed an inline policy for a user or role. Refer to the -[AWS IAM documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_managed-vs-inline.html) for the -differences between managed and inline policies. +Existing cluster profiles that use the manifest-based Cluster Autoscaler pack version 1.28.x or earlier cannot be +upgraded directly to version 1.29.x of the pack based on Helm. To use version 1.29.x, you must first remove the old +version of the pack from the cluster profile and then add the new one. ::: -4. In the cluster deployment wizard, when you are in the **Nodes configuration** section, you must enter the minimum and - maximum number of worker pool nodes, and the instance type (size) that suits your requirement. - -You must provide the node count limits because the Cluster Autoscaler uses an Auto Scaling Group to manage the cluster's -node group. An Auto Scaling Group requires a minimum and maximum count and the selection of an instance type. You can -choose an instance type that suits your requirement. - -For example, the snapshot below displays the cluster's minimum and maximum capacity. - -![A snapshot displaying the minimum and maximum node count in Palette.](/integrations_aws-cluster-autoscaler_node-count.webp) +### Parameters + +| **Parameter** | **Description** | **Default Value** | **Required** | +| ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ | ------------ | +| `charts.cluster-autoscaler.awsRegion` | The AWS region where the resources will be deployed. | `us-east-1` | Yes | +| `charts.cluster-autoscaler.autoDiscovery.clusterName` | This parameter enables autodiscovery within the EKS host cluster. | `{{ .spectro.system.cluster.name }}` | Yes | +| `charts.cluster-autoscaler.cloudProvider` | The cloud provider where the autoscaler will be deployed. | `aws` | Yes | +| `charts.cluster-autoscaler.extraArgs.expander` | Indicates which Auto Scaling Group (ASG) to expand. Options are `random`, `most-pods`, and `least-waste`. `random` scales up a random ASG. `most-pods` scales up the ASG that will schedule the most amount of pods. `least-waste` scales up the ASG that will waste the least amount of CPU/MEM resources. | `least-waste` | Yes | + +### Usage + +The Helm-based Cluster Autoscaler pack is available for Amazon EKS host clusters. To deploy the pack, you must first +define an IAM policy in the AWS account associated with Palette. This policy allows the Cluster Autoscaler to scale the +cluster's node groups. + +Use the following steps to create the IAM policy and deploy the Cluster Autoscaler pack. + +1. In AWS, create a new IAM policy using the snippet below and give it a name, for example, + _PaletteEKSClusterAutoscaler_. Refer to the + [Creating IAM policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html) guide for + instructions. + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "autoscaling:DescribeAutoScalingGroups", + "autoscaling:DescribeAutoScalingInstances", + "autoscaling:DescribeLaunchConfigurations", + "autoscaling:DescribeScalingActivities", + "autoscaling:DescribeTags", + "ec2:DescribeInstanceTypes", + "ec2:DescribeLaunchTemplateVersions" + ], + "Resource": ["*"] + }, + { + "Effect": "Allow", + "Action": [ + "autoscaling:SetDesiredCapacity", + "autoscaling:TerminateInstanceInAutoScalingGroup", + "ec2:DescribeImages", + "ec2:GetInstanceTypesFromInstanceRequirements", + "eks:DescribeNodegroup" + ], + "Resource": ["*"] + } + ] + } + ``` + +2. Copy the IAM policy + [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html). Your policy ARN + should be similar to `arn:aws:iam::123456789:policy/PaletteEKSClusterAutoscaler`. + +3. During the cluster profile creation, modify the `managedMachinePool.roleAdditionalPolicies` section in the + **values.yaml** file of the Kubernetes pack with the created IAM policy ARN. Palette will attach the IAM policy to + your cluster's node group during cluster deployment. The snapshot below illustrates the specific section to update + with the policy ARN. + + ![A snapshot displaying the ARN added to the Kubernetes pack's manifest.](/integrations_aws-cluster-autoscaler_k8s-manifest.webp) + + For example, the code block below displays the updated `managedMachinePool.roleAdditionalPolicies` section with a + sample policy ARN. + + ```yaml + managedMachinePool: + # roleName: {{ name of the self-managed role | format "${string}" }} + # A list of additional policies to attach to the node group role + roleAdditionalPolicies: + - "arn:aws:iam::123456789:policy/PaletteEKSClusterAutoscaler" + ``` + + :::tip + + Instead of updating the Kubernetes pack's **values.yaml** file, you can alternatively add an inline IAM policy to the + cluster's node group post deployment. Refer to the + [Adding IAM identity permissions](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html#add-policies-console) + guide to learn how to embed an inline policy for a user or role. + + ::: + +4. Once you included all the [infrastructure pack layers](../profiles/profiles.md) to your cluster profile, add the AWS + Cluster Autoscaler pack. + + :::warning + + The **values.yaml** file of the Cluster Autoscaler pack includes a section for setting the minimum and maximum size + of the autoscaling groups. However, this section should not be used, and this configuration must be done from the + Palette UI, according to step **5** of this guide. + + ::: + +5. Next, use the created cluster profile to [deploy a cluster](../clusters/public-cloud/deploy-k8s-cluster.md). In the + **Nodes Config** section, specify the minimum and maximum number of worker pool nodes and the instance type that + suits your workload. Your worker pool will have at least the minimum number of nodes you set up, and when scaling up, + it will not exceed the maximum number of nodes configured. Note that each configured node pool will represent one + ASG. + + The snapshot below displays an example of the cluster's **Nodes Config** section. + + ![A snapshot displaying the minimum and maximum node count in Palette.](/integrations_aws-cluster-autoscaler_node-count.webp) + + :::tip + + You can also [edit the minimum and maximum number of worker pool nodes](#resize-the-cluster) for a deployed cluster + directly in the Palette UI. + + ::: + +#### Resize the Cluster + +In this example, you will resize the worker pool nodes to better understand the scaling behavior of the Cluster +Autoscaler pack. First, you will create a cluster with large-sized worker pool instances. Then, you will manually reduce +the instance size. This will lead to insufficient resources for existing pods and multiple pod failures in the cluster. +As a result, the Cluster Autoscaler will provision new smaller-sized nodes with enough capacity to accommodate the +current workload and reschedule the affected pods on new nodes. Follow the steps below to trigger the Cluster Autoscaler +and the pod rescheduling event. + +1. During the cluster deployment, in the **Nodes Config** section, choose a large-sized instance type. For example, you + can select the worker pool instance size as **t3.2xlarge** (8 vCPUs, 32 GB RAM) or higher. + +2. Once the cluster is deployed, go to the **Nodes** tab in the cluster details page in Palette. Observe the count and + size of nodes. The snapshot below displays one node of the type **t3.2xlarge** in the cluster's worker pool. + + ![A snapshot displaying one node of the type **t3.2xlarge** in the worker pool.](/integrations_aws-cluster-autoscaler_one-node.webp) + +3. Manually reduce the instance size in the worker pool configuration to **t3.medium** (2 vCPUs, 8 GB RAM). The snapshot + below shows how to change the instance size in the node pool configuration. -
+ ![A snapshot displaying how to edit node pool configuration.](/integrations_aws-cluster-autoscaler_edit-node.webp) -### Resize the Cluster - -To better understand the scaling behavior of the Cluster Autoscaler and its impact on a cluster, do the following -exercise to gain firsthand experience with the scaling behavior. - -In the following example scenario, you will first create a cluster with large-sized worker pool instances. Next, you -will manually reduce the instance size, leading to insufficient resources for existing pods and multiple pod failures in -the cluster. As a result, the Cluster Autoscaler will provision new smaller-sized nodes with enough capacity to -accommodate the current workload and reschedule those contending pods on new nodes. Also, the new nodes' count will be -within the minimum and maximum limit you specified for the worker pool. +4. Wait a few minutes for the new nodes to be provisioned. Reducing the node size will make the Cluster Autoscaler shut + down the large node and provision smaller-sized nodes with enough capacity to accommodate the current workload. Note + that the new node count will be within the minimum and maximum limits specified in the worker pool configuration + wizard. -Use the following steps to trigger the pod rescheduling event manually: + The snapshot below displays the newly created **t3.medium** nodes. These two smaller-sized nodes will efficiently + manage the same workload as the single larger-sized node. -
+ ![A snapshot displaying new nodes of the size **t3.medium** spin up automatically, *collectively* providing enough capacity to accommodate the current workload. ](/integrations_aws-cluster-autoscaler_two-nodes.webp) -1. In the cluster deployment wizard, while defining the **Nodes configuration**, choose a large-sized instance type. For - example, you can choose your worker pool to have instance size **t3.2xlarge** (8 vCPUs, 32 GB RAM) or higher. + -2. After your cluster is successfully deployed, navigate to the **Nodes** tab in the cluster details page in Palette, - and note the count and size of nodes. The snapshots below display one node of the type **t3.2xlarge** in the worker - pool of a successfully deployed cluster. + -![A snapshot displaying one node of the type **t3.2xlarge** in the worker pool.](/integrations_aws-cluster-autoscaler_one-node.webp) +### Prerequisites -3. Manually reduce the instance size in the worker-pool configuration to a **t3.medium** (2 vCPUs, 8 GB RAM). The - snapshot below displays how to edit the instance size in the node pool configuration. +- An EKS host cluster with Kubernetes 1.28.x or higher. -![A snapshot displaying how to edit node pool configuration.](/integrations_aws-cluster-autoscaler_edit-node.webp) +- Permission to create an IAM policy in the AWS account you use with Palette. -4. Wait for a few minutes for the new nodes to provision. Reducing the node size will make the Cluster Autoscaler shut - down the large node and provision smaller-sized nodes with enough capacity to accommodate the current workload. Also, - the new node count will be within the minimum and maximum limit you specified for the worker pool configuration +### Parameters + +| **Parameter** | **Description** | **Default Value** | **Required** | +| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | ------------ | +| `manifests.aws-cluster-autoscaler.expander` | Indicates which Auto Scaling Group (ASG) to expand. Options are `random`, `most-pods`, and `least-waste`. `random` scales up a random ASG. `most-pods` scales up the ASG that will schedule the most amount of pods. `least-waste` scales up the ASG that will waste the least amount of CPU/MEM resources. | `least-waste` | Yes | + +### Usage + +The manifest-based Cluster Autoscaler pack is available for Amazon EKS host clusters. To deploy the pack, you must first +define an IAM policy in the AWS account associated with Palette. This policy allows the Cluster Autoscaler to scale the +cluster's node groups. + +Use the following steps to create the IAM policy and deploy the Cluster Autoscaler pack. + +1. In AWS, create a new IAM policy using the snippet below and give it a name, for example, + _PaletteEKSClusterAutoscaler_. Refer to the + [Creating IAM policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html) guide for + instructions. + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "autoscaling:DescribeAutoScalingGroups", + "autoscaling:DescribeAutoScalingInstances", + "autoscaling:DescribeLaunchConfigurations", + "autoscaling:DescribeScalingActivities", + "autoscaling:DescribeTags", + "ec2:DescribeInstanceTypes", + "ec2:DescribeLaunchTemplateVersions" + ], + "Resource": ["*"] + }, + { + "Effect": "Allow", + "Action": [ + "autoscaling:SetDesiredCapacity", + "autoscaling:TerminateInstanceInAutoScalingGroup", + "ec2:DescribeImages", + "ec2:GetInstanceTypesFromInstanceRequirements", + "eks:DescribeNodegroup" + ], + "Resource": ["*"] + } + ] + } + ``` + +2. Copy the IAM policy + [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html). Your policy ARN + should be similar to `arn:aws:iam::123456789:policy/PaletteEKSClusterAutoscaler`. + +3. During the cluster profile creation, modify the `managedMachinePool.roleAdditionalPolicies` section in the + **values.yaml** file of the Kubernetes pack with the created IAM policy ARN. Palette will attach the IAM policy to + your cluster's node group during cluster deployment. The snapshot below illustrates the specific section to update + with the policy ARN. + + ![A snapshot displaying the ARN added to the Kubernetes pack's manifest.](/integrations_aws-cluster-autoscaler_k8s-manifest.webp) + + For example, the code block below displays the updated `managedMachinePool.roleAdditionalPolicies` section with a + sample policy ARN. + + ```yaml + managedMachinePool: + # roleName: {{ name of the self-managed role | format "${string}" }} + # A list of additional policies to attach to the node group role + roleAdditionalPolicies: + - "arn:aws:iam::123456789:policy/PaletteEKSClusterAutoscaler" + ``` + + :::tip + + Instead of updating the Kubernetes pack's **values.yaml** file, you can alternatively add an inline IAM policy to the + cluster's node group post deployment. Refer to the + [Adding IAM identity permissions](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html#add-policies-console) + guide to learn how to embed an inline policy for a user or role. + + ::: + +4. Once you included all the [infrastructure pack layers](../profiles/profiles.md) to your cluster profile, add the AWS + Cluster Autoscaler pack. + +5. Next, use the created cluster profile to [deploy a cluster](../clusters/public-cloud/deploy-k8s-cluster.md). In the + **Nodes Config** section, specify the minimum and maximum number of worker pool nodes and the instance type that + suits your workload. Your worker pool will have at least the minimum number of nodes you set up, and when scaling up, + it will not exceed the maximum number of nodes configured. Note that each configured node pool will represent one + ASG. + + The snapshot below displays an example of the cluster's **Nodes Config** section. + + ![A snapshot displaying the minimum and maximum node count in Palette.](/integrations_aws-cluster-autoscaler_node-count.webp) + + :::tip + + You can also [edit the minimum and maximum number of worker pool nodes](#resize-the-cluster) for a deployed cluster + directly in the Palette UI. + + ::: + +#### Resize the Cluster + +In this example, you will resize the worker pool nodes to better understand the scaling behavior of the Cluster +Autoscaler pack. First, you will create a cluster with large-sized worker pool instances. Then, you will manually reduce +the instance size. This will lead to insufficient resources for existing pods and multiple pod failures in the cluster. +As a result, the Cluster Autoscaler will provision new smaller-sized nodes with enough capacity to accommodate the +current workload and reschedule the affected pods on new nodes. Follow the steps below to trigger the Cluster Autoscaler +and the pod rescheduling event. + +1. During the cluster deployment, in the **Nodes Config** section, choose a large-sized instance type. For example, you + can select the worker pool instance size as **t3.2xlarge** (8 vCPUs, 32 GB RAM) or higher. + +2. Once the cluster is deployed, go to the **Nodes** tab in the cluster details page in Palette. Observe the count and + size of nodes. The snapshot below displays one node of the type **t3.2xlarge** in the cluster's worker pool. + + ![A snapshot displaying one node of the type **t3.2xlarge** in the worker pool.](/integrations_aws-cluster-autoscaler_one-node.webp) + +3. Manually reduce the instance size in the worker pool configuration to **t3.medium** (2 vCPUs, 8 GB RAM). The snapshot + below shows how to change the instance size in the node pool configuration. + + ![A snapshot displaying how to edit node pool configuration.](/integrations_aws-cluster-autoscaler_edit-node.webp) + +4. Wait a few minutes for the new nodes to be provisioned. Reducing the node size will make the Cluster Autoscaler shut + down the large node and provision smaller-sized nodes with enough capacity to accommodate the current workload. Note + that the new node count will be within the minimum and maximum limits specified in the worker pool configuration wizard. -The following snapshot displays two new nodes of the size **t3.medium** spin up automatically. These two smaller-sized -nodes will be able to handle the same workload as a single larger-sized node. - -![A snapshot displaying new nodes of the size **t3.medium** spin up automatically, *collectively* providing enough capacity to accommodate the current workload. ](/integrations_aws-cluster-autoscaler_two-nodes.webp) + The snapshot below displays the newly created **t3.medium** nodes. These two smaller-sized nodes will efficiently + manage the same workload as the single larger-sized node. -
+ ![A snapshot displaying new nodes of the size **t3.medium** spin up automatically, *collectively* providing enough capacity to accommodate the current workload. ](/integrations_aws-cluster-autoscaler_two-nodes.webp) - + -## Prerequisites +### Prerequisites -- Kubernetes 1.19.x or higher. +- An EKS host cluster with Kubernetes 1.27.x or higher. - Permission to create an IAM policy in the AWS account you use with Palette. -- IAM policy - A - [Full Cluster Autoscaler Features](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md#full-cluster-autoscaler-features-policy-recommended) - IAM policy must be attached to the EKS cluster's node group. The policy must allow the Cluster Autoscaler to scale the - cluster's node groups. - - There are two ways to achieve this prerequisite. You can define the policy as a _customer-managed_ policy in the AWS - account and use its Amazon Resource Name (ARN) in the cluster profile. Alternatively, you can attach the IAM policy as - an _inline_ policy to the node group if you have already deployed your cluster. Refer to the [Usage](#usage) section - below to learn more. - -- Updated Kubernetes layer manifest - The Kubernetes pack's manifest should be updated with the newly created IAM policy - ARN. The YAML code block below displays the `managedMachinePool.roleAdditionalPolicies` section to update in the - Kubernetes pack's manifest. Refer to the [Usage](#usage) section below for more details with an example. - -
- - ```yaml - managedMachinePool: - #roleName: {{ name of the self-managed role | format "${string}" }} - ## A list of additional policies to attach to the node group role - roleAdditionalPolicies: - - {{ arn for the policy1 | format "${string}" }} - ``` - -
- -## Usage - -Cluster Autoscaler helps improve your cluster's performance and makes your cluster more resilient to failures. It -automatically adjusts the number of nodes in your cluster based on the current workload. In other words, Cluster -Autoscaler monitors the resource utilization, such as CPU and memory, and the number of pods active in your cluster and -scales the cluster when either of these events occurs: - -
- -- Multiple pods fail due to resource contention. In this case, the Cluster Autoscaler will provision more nodes. - -- Nodes are underutilized for a specific period. In this case, the Cluster Autoscaler will reschedule the pods onto - other nodes and shut down the underutilized node. -
- -### Deploy Cluster Autoscaler - -To deploy the Cluster Autoscaler pack, you must first define an IAM policy in the AWS account associated with Palette. - -Next, update the cluster profile to specify the IAM policy ARN in the Kubernetes pack's manifest. Palette will attach -that IAM policy to your cluster's node group during deployment. Note that Palette automatically creates two IAM roles in -the AWS account when you deploy an EKS cluster. One role is for the cluster, and another for the cluster's node group. -The cluster's IAM role name will have the following naming convention, `[your-cluster-name]-iam-service-role`, and the -node group's IAM role name will follow the `ng-role_worker-pool-[random-string]` naming convention. - -The following steps provide detailed instructions for deploying the Cluster Autoscaler pack. - -
- -1. Define the new IAM policy using the policy outlined below, and give it a name, for example, - _PaletteEKSClusterAutoscaler_. -
- -```json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Action": [ - "autoscaling:DescribeAutoScalingGroups", - "autoscaling:DescribeAutoScalingInstances", - "autoscaling:DescribeLaunchConfigurations", - "autoscaling:DescribeScalingActivities", - "autoscaling:DescribeTags", - "ec2:DescribeInstanceTypes", - "ec2:DescribeLaunchTemplateVersions" - ], - "Resource": ["*"] - }, - { - "Effect": "Allow", - "Action": [ - "autoscaling:SetDesiredCapacity", - "autoscaling:TerminateInstanceInAutoScalingGroup", - "ec2:DescribeImages", - "ec2:GetInstanceTypesFromInstanceRequirements", - "eks:DescribeNodegroup" - ], - "Resource": ["*"] - } - ] -} -``` - -2. Copy the IAM policy ARN to the clipboard for the next step. For example, the policy ARN will be similar to - `arn:aws:iam::650628870702:policy/PaletteEKSClusterAutoscaler`. - -3. In your cluster profile, and update the `managedMachinePool.roleAdditionalPolicies` section in the Kubernetes pack's - manifest with the newly created IAM policy ARN. The snapshot below displays the specific section to update with the - policy ARN. - -![A snapshot displaying the ARN added to the Kubernetes pack's manifest.](/integrations_aws-cluster-autoscaler_k8s-manifest.webp) - -For example, the code block below displays the updated `managedMachinePool.roleAdditionalPolicies` section with a sample -policy ARN, `"arn:aws:iam::650628870702:policy/PaletteEKSClusterAutoscaler"`. Before you use the following code block, -replace the ARN below with yours. - -
- -```yaml -managedMachinePool: - # roleName: {{ name of the self-managed role | format "${string}" }} - # A list of additional policies to attach to the node group role - roleAdditionalPolicies: - - "arn:aws:iam::650628870702:policy/PaletteEKSClusterAutoscaler" -``` +### Parameters + +| **Parameter** | **Description** | **Default Value** | **Required** | +| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | ------------ | +| `manifests.aws-cluster-autoscaler.expander` | Indicates which Auto Scaling Group (ASG) to expand. Options are `random`, `most-pods`, and `least-waste`. `random` scales up a random ASG. `most-pods` scales up the ASG that will schedule the most amount of pods. `least-waste` scales up the ASG that will waste the least amount of CPU/MEM resources. | `least-waste` | Yes | + +### Usage + +The manifest-based Cluster Autoscaler pack is available for Amazon EKS host clusters. To deploy the pack, you must first +define an IAM policy in the AWS account associated with Palette. This policy allows the Cluster Autoscaler to scale the +cluster's node groups. + +Use the following steps to create the IAM policy and deploy the Cluster Autoscaler pack. + +1. In AWS, create a new IAM policy using the snippet below and give it a name, for example, + _PaletteEKSClusterAutoscaler_. Refer to the + [Creating IAM policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html) guide for + instructions. + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "autoscaling:DescribeAutoScalingGroups", + "autoscaling:DescribeAutoScalingInstances", + "autoscaling:DescribeLaunchConfigurations", + "autoscaling:DescribeScalingActivities", + "autoscaling:DescribeTags", + "ec2:DescribeInstanceTypes", + "ec2:DescribeLaunchTemplateVersions" + ], + "Resource": ["*"] + }, + { + "Effect": "Allow", + "Action": [ + "autoscaling:SetDesiredCapacity", + "autoscaling:TerminateInstanceInAutoScalingGroup", + "ec2:DescribeImages", + "ec2:GetInstanceTypesFromInstanceRequirements", + "eks:DescribeNodegroup" + ], + "Resource": ["*"] + } + ] + } + ``` + +2. Copy the IAM policy + [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html). Your policy ARN + should be similar to `arn:aws:iam::123456789:policy/PaletteEKSClusterAutoscaler`. + +3. During the cluster profile creation, modify the `managedMachinePool.roleAdditionalPolicies` section in the + **values.yaml** file of the Kubernetes pack with the created IAM policy ARN. Palette will attach the IAM policy to + your cluster's node group during cluster deployment. The snapshot below illustrates the specific section to update + with the policy ARN. + + ![A snapshot displaying the ARN added to the Kubernetes pack's manifest.](/integrations_aws-cluster-autoscaler_k8s-manifest.webp) + + For example, the code block below displays the updated `managedMachinePool.roleAdditionalPolicies` section with a + sample policy ARN. + + ```yaml + managedMachinePool: + # roleName: {{ name of the self-managed role | format "${string}" }} + # A list of additional policies to attach to the node group role + roleAdditionalPolicies: + - "arn:aws:iam::123456789:policy/PaletteEKSClusterAutoscaler" + ``` + + :::tip + + Instead of updating the Kubernetes pack's **values.yaml** file, you can alternatively add an inline IAM policy to the + cluster's node group post deployment. Refer to the + [Adding IAM identity permissions](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html#add-policies-console) + guide to learn how to embed an inline policy for a user or role. + + ::: + +4. Once you included all the [infrastructure pack layers](../profiles/profiles.md) to your cluster profile, add the AWS + Cluster Autoscaler pack. + +5. Next, use the created cluster profile to [deploy a cluster](../clusters/public-cloud/deploy-k8s-cluster.md). In the + **Nodes Config** section, specify the minimum and maximum number of worker pool nodes and the instance type that + suits your workload. Your worker pool will have at least the minimum number of nodes you set up, and when scaling up, + it will not exceed the maximum number of nodes configured. Note that each configured node pool will represent one + ASG. + + The snapshot below displays an example of the cluster's **Nodes Config** section. + + ![A snapshot displaying the minimum and maximum node count in Palette.](/integrations_aws-cluster-autoscaler_node-count.webp) + + :::tip + + You can also [edit the minimum and maximum number of worker pool nodes](#resize-the-cluster) for a deployed cluster + directly in the Palette UI. + + ::: + +#### Resize the Cluster + +In this example, you will resize the worker pool nodes to better understand the scaling behavior of the Cluster +Autoscaler pack. First, you will create a cluster with large-sized worker pool instances. Then, you will manually reduce +the instance size. This will lead to insufficient resources for existing pods and multiple pod failures in the cluster. +As a result, the Cluster Autoscaler will provision new smaller-sized nodes with enough capacity to accommodate the +current workload and reschedule the affected pods on new nodes. Follow the steps below to trigger the Cluster Autoscaler +and the pod rescheduling event. + +1. During the cluster deployment, in the **Nodes Config** section, choose a large-sized instance type. For example, you + can select the worker pool instance size as **t3.2xlarge** (8 vCPUs, 32 GB RAM) or higher. + +2. Once the cluster is deployed, go to the **Nodes** tab in the cluster details page in Palette. Observe the count and + size of nodes. The snapshot below displays one node of the type **t3.2xlarge** in the cluster's worker pool. + + ![A snapshot displaying one node of the type **t3.2xlarge** in the worker pool.](/integrations_aws-cluster-autoscaler_one-node.webp) + +3. Manually reduce the instance size in the worker pool configuration to **t3.medium** (2 vCPUs, 8 GB RAM). The snapshot + below shows how to change the instance size in the node pool configuration. + + ![A snapshot displaying how to edit node pool configuration.](/integrations_aws-cluster-autoscaler_edit-node.webp) + +4. Wait a few minutes for the new nodes to be provisioned. Reducing the node size will make the Cluster Autoscaler shut + down the large node and provision smaller-sized nodes with enough capacity to accommodate the current workload. Note + that the new node count will be within the minimum and maximum limits specified in the worker pool configuration + wizard. -
+ The snapshot below displays the newly created **t3.medium** nodes. These two smaller-sized nodes will efficiently + manage the same workload as the single larger-sized node. -:::info + ![A snapshot displaying new nodes of the size **t3.medium** spin up automatically, *collectively* providing enough capacity to accommodate the current workload. ](/integrations_aws-cluster-autoscaler_two-nodes.webp) -If you do not want to update the Kubernetes pack's manifest, you can add an _inline_ IAM policy to the cluster's node -group post deployment. Refer to this -[AWS guide](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html#add-policies-console) -on how to embed an inline policy for a user or role. Refer to the -[AWS IAM documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_managed-vs-inline.html) for the -differences between managed and inline policies. + -::: + -4. In the cluster deployment wizard, when you are in the **Nodes configuration** section, you must enter the minimum and - maximum number of worker pool nodes, and the instance type (size) that suits your requirement. +### Prerequisites -You must provide the node count limits because the Cluster Autoscaler uses an Auto Scaling Group to manage the cluster's -node group. An Auto Scaling Group requires a minimum and maximum count and the selection of an instance type. You can -choose an instance type that suits your requirement. +- An EKS host cluster with Kubernetes 1.26.x or higher. -For example, the snapshot below displays the cluster's minimum and maximum capacity. +- Permission to create an IAM policy in the AWS account you use with Palette. -![A snapshot displaying the minimum and maximum node count in Palette.](/integrations_aws-cluster-autoscaler_node-count.webp) +### Parameters + +| **Parameter** | **Description** | **Default Value** | **Required** | +| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | ------------ | +| `manifests.aws-cluster-autoscaler.expander` | Indicates which Auto Scaling Group (ASG) to expand. Options are `random`, `most-pods`, and `least-waste`. `random` scales up a random ASG. `most-pods` scales up the ASG that will schedule the most amount of pods. `least-waste` scales up the ASG that will waste the least amount of CPU/MEM resources. | `least-waste` | Yes | + +### Usage + +The manifest-based Cluster Autoscaler pack is available for Amazon EKS host clusters. To deploy the pack, you must first +define an IAM policy in the AWS account associated with Palette. This policy allows the Cluster Autoscaler to scale the +cluster's node groups. + +Use the following steps to create the IAM policy and deploy the Cluster Autoscaler pack. + +1. In AWS, create a new IAM policy using the snippet below and give it a name, for example, + _PaletteEKSClusterAutoscaler_. Refer to the + [Creating IAM policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html) guide for + instructions. + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "autoscaling:DescribeAutoScalingGroups", + "autoscaling:DescribeAutoScalingInstances", + "autoscaling:DescribeLaunchConfigurations", + "autoscaling:DescribeScalingActivities", + "autoscaling:DescribeTags", + "ec2:DescribeInstanceTypes", + "ec2:DescribeLaunchTemplateVersions" + ], + "Resource": ["*"] + }, + { + "Effect": "Allow", + "Action": [ + "autoscaling:SetDesiredCapacity", + "autoscaling:TerminateInstanceInAutoScalingGroup", + "ec2:DescribeImages", + "ec2:GetInstanceTypesFromInstanceRequirements", + "eks:DescribeNodegroup" + ], + "Resource": ["*"] + } + ] + } + ``` + +2. Copy the IAM policy + [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html). Your policy ARN + should be similar to `arn:aws:iam::123456789:policy/PaletteEKSClusterAutoscaler`. + +3. During the cluster profile creation, modify the `managedMachinePool.roleAdditionalPolicies` section in the + **values.yaml** file of the Kubernetes pack with the created IAM policy ARN. Palette will attach the IAM policy to + your cluster's node group during cluster deployment. The snapshot below illustrates the specific section to update + with the policy ARN. + + ![A snapshot displaying the ARN added to the Kubernetes pack's manifest.](/integrations_aws-cluster-autoscaler_k8s-manifest.webp) + + For example, the code block below displays the updated `managedMachinePool.roleAdditionalPolicies` section with a + sample policy ARN. + + ```yaml + managedMachinePool: + # roleName: {{ name of the self-managed role | format "${string}" }} + # A list of additional policies to attach to the node group role + roleAdditionalPolicies: + - "arn:aws:iam::123456789:policy/PaletteEKSClusterAutoscaler" + ``` + + :::tip + + Instead of updating the Kubernetes pack's **values.yaml** file, you can alternatively add an inline IAM policy to the + cluster's node group post deployment. Refer to the + [Adding IAM identity permissions](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html#add-policies-console) + guide to learn how to embed an inline policy for a user or role. + + ::: + +4. Once you included all the [infrastructure pack layers](../profiles/profiles.md) to your cluster profile, add the AWS + Cluster Autoscaler pack. + +5. Next, use the created cluster profile to [deploy a cluster](../clusters/public-cloud/deploy-k8s-cluster.md). In the + **Nodes Config** section, specify the minimum and maximum number of worker pool nodes and the instance type that + suits your workload. Your worker pool will have at least the minimum number of nodes you set up, and when scaling up, + it will not exceed the maximum number of nodes configured. Note that each configured node pool will represent one + ASG. + + The snapshot below displays an example of the cluster's **Nodes Config** section. + + ![A snapshot displaying the minimum and maximum node count in Palette.](/integrations_aws-cluster-autoscaler_node-count.webp) + + :::tip + + You can also [edit the minimum and maximum number of worker pool nodes](#resize-the-cluster) for a deployed cluster + directly in the Palette UI. + + ::: + +#### Resize the Cluster + +In this example, you will resize the worker pool nodes to better understand the scaling behavior of the Cluster +Autoscaler pack. First, you will create a cluster with large-sized worker pool instances. Then, you will manually reduce +the instance size. This will lead to insufficient resources for existing pods and multiple pod failures in the cluster. +As a result, the Cluster Autoscaler will provision new smaller-sized nodes with enough capacity to accommodate the +current workload and reschedule the affected pods on new nodes. Follow the steps below to trigger the Cluster Autoscaler +and the pod rescheduling event. + +1. During the cluster deployment, in the **Nodes Config** section, choose a large-sized instance type. For example, you + can select the worker pool instance size as **t3.2xlarge** (8 vCPUs, 32 GB RAM) or higher. + +2. Once the cluster is deployed, go to the **Nodes** tab in the cluster details page in Palette. Observe the count and + size of nodes. The snapshot below displays one node of the type **t3.2xlarge** in the cluster's worker pool. + + ![A snapshot displaying one node of the type **t3.2xlarge** in the worker pool.](/integrations_aws-cluster-autoscaler_one-node.webp) + +3. Manually reduce the instance size in the worker pool configuration to **t3.medium** (2 vCPUs, 8 GB RAM). The snapshot + below shows how to change the instance size in the node pool configuration. + + ![A snapshot displaying how to edit node pool configuration.](/integrations_aws-cluster-autoscaler_edit-node.webp) + +4. Wait a few minutes for the new nodes to be provisioned. Reducing the node size will make the Cluster Autoscaler shut + down the large node and provision smaller-sized nodes with enough capacity to accommodate the current workload. Note + that the new node count will be within the minimum and maximum limits specified in the worker pool configuration + wizard. -
+ The snapshot below displays the newly created **t3.medium** nodes. These two smaller-sized nodes will efficiently + manage the same workload as the single larger-sized node. -### Resize the Cluster + ![A snapshot displaying new nodes of the size **t3.medium** spin up automatically, *collectively* providing enough capacity to accommodate the current workload. ](/integrations_aws-cluster-autoscaler_two-nodes.webp) -To better understand the scaling behavior of the Cluster Autoscaler and its impact on a cluster, do the following -exercise to gain firsthand experience with the scaling behavior. + -In the following example scenario, you will first create a cluster with large-sized worker pool instances. Next, you -will manually reduce the instance size, leading to insufficient resources for existing pods and multiple pod failures in -the cluster. As a result, the Cluster Autoscaler will provision new smaller-sized nodes with enough capacity to -accommodate the current workload and reschedule those contending pods on new nodes. Also, the new nodes' count will be -within the minimum and maximum limit you specified for the worker pool. + -Use the following steps to trigger the pod rescheduling event manually: +### Prerequisites -
+- An EKS host cluster with Kubernetes 1.19.x or higher. -1. In the cluster deployment wizard, while defining the **Nodes configuration**, choose a large-sized instance type. For - example, you can choose your worker pool to have instance size **t3.2xlarge** (8 vCPUs, 32 GB RAM) or higher. +- Permission to create an IAM policy in the AWS account you use with Palette. -2. After your cluster is successfully deployed, navigate to the **Nodes** tab in the cluster details page in Palette, - and note the count and size of nodes. The snapshots below display one node of the type **t3.2xlarge** in the worker - pool of a successfully deployed cluster. +### Parameters + +| **Parameter** | **Description** | **Default Value** | **Required** | +| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | ------------ | +| `manifests.aws-cluster-autoscaler.expander` | Indicates which Auto Scaling Group (ASG) to expand. Options are `random`, `most-pods`, and `least-waste`. `random` scales up a random ASG. `most-pods` scales up the ASG that will schedule the most amount of pods. `least-waste` scales up the ASG that will waste the least amount of CPU/MEM resources. | `least-waste` | Yes | + +### Usage + +The manifest-based Cluster Autoscaler pack is available for Amazon EKS host clusters. To deploy the pack, you must first +define an IAM policy in the AWS account associated with Palette. This policy allows the Cluster Autoscaler to scale the +cluster's node groups. + +Use the following steps to create the IAM policy and deploy the Cluster Autoscaler pack. + +1. In AWS, create a new IAM policy using the snippet below and give it a name, for example, + _PaletteEKSClusterAutoscaler_. Refer to the + [Creating IAM policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create.html) guide for + instructions. + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "autoscaling:DescribeAutoScalingGroups", + "autoscaling:DescribeAutoScalingInstances", + "autoscaling:DescribeLaunchConfigurations", + "autoscaling:DescribeScalingActivities", + "autoscaling:DescribeTags", + "ec2:DescribeInstanceTypes", + "ec2:DescribeLaunchTemplateVersions" + ], + "Resource": ["*"] + }, + { + "Effect": "Allow", + "Action": [ + "autoscaling:SetDesiredCapacity", + "autoscaling:TerminateInstanceInAutoScalingGroup", + "ec2:DescribeImages", + "ec2:GetInstanceTypesFromInstanceRequirements", + "eks:DescribeNodegroup" + ], + "Resource": ["*"] + } + ] + } + ``` + +2. Copy the IAM policy + [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html). Your policy ARN + should be similar to `arn:aws:iam::123456789:policy/PaletteEKSClusterAutoscaler`. + +3. During the cluster profile creation, modify the `managedMachinePool.roleAdditionalPolicies` section in the + **values.yaml** file of the Kubernetes pack with the created IAM policy ARN. Palette will attach the IAM policy to + your cluster's node group during cluster deployment. The snapshot below illustrates the specific section to update + with the policy ARN. + + ![A snapshot displaying the ARN added to the Kubernetes pack's manifest.](/integrations_aws-cluster-autoscaler_k8s-manifest.webp) + + For example, the code block below displays the updated `managedMachinePool.roleAdditionalPolicies` section with a + sample policy ARN. + + ```yaml + managedMachinePool: + # roleName: {{ name of the self-managed role | format "${string}" }} + # A list of additional policies to attach to the node group role + roleAdditionalPolicies: + - "arn:aws:iam::123456789:policy/PaletteEKSClusterAutoscaler" + ``` + + :::tip + + Instead of updating the Kubernetes pack's **values.yaml** file, you can alternatively add an inline IAM policy to the + cluster's node group post deployment. Refer to the + [Adding IAM identity permissions](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html#add-policies-console) + guide to learn how to embed an inline policy for a user or role. + + ::: + +4. Once you included all the [infrastructure pack layers](../profiles/profiles.md) to your cluster profile, add the AWS + Cluster Autoscaler pack. + +5. Next, use the created cluster profile to [deploy a cluster](../clusters/public-cloud/deploy-k8s-cluster.md). In the + **Nodes Config** section, specify the minimum and maximum number of worker pool nodes and the instance type that + suits your workload. Your worker pool will have at least the minimum number of nodes you set up, and when scaling up, + it will not exceed the maximum number of nodes configured. Note that each configured node pool will represent one + ASG. + + The snapshot below displays an example of the cluster's **Nodes Config** section. + + ![A snapshot displaying the minimum and maximum node count in Palette.](/integrations_aws-cluster-autoscaler_node-count.webp) + + :::tip + + You can also [edit the minimum and maximum number of worker pool nodes](#resize-the-cluster) for a deployed cluster + directly in the Palette UI. + + ::: + +#### Resize the Cluster + +In this example, you will resize the worker pool nodes to better understand the scaling behavior of the Cluster +Autoscaler pack. First, you will create a cluster with large-sized worker pool instances. Then, you will manually reduce +the instance size. This will lead to insufficient resources for existing pods and multiple pod failures in the cluster. +As a result, the Cluster Autoscaler will provision new smaller-sized nodes with enough capacity to accommodate the +current workload and reschedule the affected pods on new nodes. Follow the steps below to trigger the Cluster Autoscaler +and the pod rescheduling event. + +1. During the cluster deployment, in the **Nodes Config** section, choose a large-sized instance type. For example, you + can select the worker pool instance size as **t3.2xlarge** (8 vCPUs, 32 GB RAM) or higher. + +2. Once the cluster is deployed, go to the **Nodes** tab in the cluster details page in Palette. Observe the count and + size of nodes. The snapshot below displays one node of the type **t3.2xlarge** in the cluster's worker pool. + + ![A snapshot displaying one node of the type **t3.2xlarge** in the worker pool.](/integrations_aws-cluster-autoscaler_one-node.webp) + +3. Manually reduce the instance size in the worker pool configuration to **t3.medium** (2 vCPUs, 8 GB RAM). The snapshot + below shows how to change the instance size in the node pool configuration. + + ![A snapshot displaying how to edit node pool configuration.](/integrations_aws-cluster-autoscaler_edit-node.webp) + +4. Wait a few minutes for the new nodes to be provisioned. Reducing the node size will make the Cluster Autoscaler shut + down the large node and provision smaller-sized nodes with enough capacity to accommodate the current workload. Note + that the new node count will be within the minimum and maximum limits specified in the worker pool configuration + wizard. -![A snapshot displaying one node of the type **t3.2xlarge** in the worker pool.](/integrations_aws-cluster-autoscaler_one-node.webp) + The snapshot below displays the newly created **t3.medium** nodes. These two smaller-sized nodes will efficiently + manage the same workload as the single larger-sized node. -3. Manually reduce the instance size in the worker-pool configuration to a **t3.medium** (2 vCPUs, 8 GB RAM). The - snapshot below displays how to edit the instance size in the node pool configuration. + ![A snapshot displaying new nodes of the size **t3.medium** spin up automatically, *collectively* providing enough capacity to accommodate the current workload. ](/integrations_aws-cluster-autoscaler_two-nodes.webp) -![A snapshot displaying how to edit node pool configuration.](/integrations_aws-cluster-autoscaler_edit-node.webp) + -4. Wait for a few minutes for the new nodes to provision. Reducing the node size will make the Cluster Autoscaler shut - down the large node and provision smaller-sized nodes with enough capacity to accommodate the current workload. Also, - the new node count will be within the minimum and maximum limit you specified for the worker pool configuration - wizard. + -The following snapshot displays two new nodes of the size **t3.medium** spin up automatically. These two smaller-sized -nodes will be able to handle the same workload as a single larger-sized node. +:::warning -![A snapshot displaying new nodes of the size **t3.medium** spin up automatically, *collectively* providing enough capacity to accommodate the current workload. ](/integrations_aws-cluster-autoscaler_two-nodes.webp) +All versions less than version 1.22.x are considered deprecated. Upgrade to a newer version to take advantage of new +features. -
+::: @@ -421,18 +787,23 @@ nodes will be able to handle the same workload as a single larger-sized node. ## Troubleshooting -If you are facing the `LimitExceeded: Cannot exceed quota for PoliciesPerRole:10` error in the cluster deployment logs, -it may be because the default IAM role Palette creates for the node group already has 10 policies attached to it, and -you are trying to attach one more. By default, your AWS account will have a quota of 10 managed policies per IAM role. -To fix the error, follow the instruction in this -[AWS guide](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-quotas.html#reference_iam-quotas-entities) to -request a quota increase. +### Scenario - Quota Limits Exceeded + +You may observe the `LimitExceeded: Cannot exceed quota for PoliciesPerRole:10` error in the cluster deployment logs. +This may happen because the default IAM role that Palette creates for the node group already has 10 policies attached, +and you are trying to attach one more. By default, your AWS account will have a quota of 10 managed policies per IAM +role. The workaround is to follow the instructions in the +[IAM object quotas](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-quotas.html#reference_iam-quotas-entities) +guide to request a quota increase. + +### Scenario - IAM Authenticator not Found -If you encounter an `executable aws-iam-authenticator not found` error in your terminal when attempting to access your -EKS cluster from your local machine, it may be due to the +You may encounter an `executable aws-iam-authenticator not found` error in your terminal when attempting to access your +EKS cluster from your local machine. This may happen due to the [aws-iam-authenticator](https://github.com/kubernetes-sigs/aws-iam-authenticator) plugin missing from your local -environment. You can find the installation steps for the aws-iam-authenticator in the following -[install guide](https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html). +environment. The workaround is to install the IAM Authenticator. Refer to the +[IAM Authenticator](https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html) install guide +for more information. ## Terraform @@ -445,7 +816,7 @@ data "spectrocloud_registry" "public_registry" { data "spectrocloud_pack_simple" "aws-cluster-autoscaler" { name = "aws-cluster-autoscaler" - version = "1.26.3" + version = "1.29.2" type = "helm" registry_uid = data.spectrocloud_registry.public_registry.id }