Short & Long term Roadmap

[sedefsavas]

What features do we want to add in the short (v1) and long term (v2+) to CAPA?

Please create an issue for each feature you propose and then add a brief summary below with the issue link. We can vote for them below to prioritize.

Note 1: Short term feature ideas should not have any breaking API changes.

Note 2: It will be great if you can add a note to the issues if you can contribute to its implementation. This will help us determine a more realistic roadmap.

[sedefsavas]

Parallelize resource creation for faster reconciliation: #2953

[richardcase]

For EKS:

• bootstrap.sh options Support additional EKS config options #2853
• pre-bootstrap commands prebootstrapcommand in machinepool / managed machinepool #2771
• Launch Templates AWSManagedMachinePool - Launch Template support #2055, EKS kubelet-extra-args are not set when using managedmachinepool #2224
• IPv6 and dual-stack support
• Stage 2 Identity Types Implement the stage 2 namespace scoped principal types #2336
• EKS e2e improvements (multiple issues)
• ClusterClass for AWSManagedControlPlane (no issue yet)

[sedefsavas]

Supporting different network topologies: #1484

[sedefsavas]

Spot instance support in MachinePools:#2523

[sedefsavas]

Also, mix usage of On-Demand and Spot Instances in ASG is supported.

[sedefsavas]

AWS VPC CNI support for Cluster API managed clusters:
#3286

[sedefsavas]

IPv6 only and dual stack support: #2420
#3381

[sedefsavas]

Add workload identity support to CAPA-managed clusters, similar to EKS IRSA (IAM roles for service accounts) support (EKS-managed CAPA clusters have this support)
#2054

[sedefsavas]

A rough EKS IRSA workflow:

• EKS creates OIDC provider
• Using service account token volume projection, JWT token is requested from the OIDC provider by kubelet and the token is automatically updated.
• This token is mounted to the pods by kubelet
• Use webhooks to set AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN environment variables in the pods.

[sedefsavas]

Use AWS Controllers for Kubernetes (ACK) for AWS SDK calls:
#3306

[sedefsavas]

Multiple VPC CIDR support/ Secondary CIDR support
#3287

[sedefsavas]

Graceful shutdown of nodes: #2574

2 cases to cover under this feature:

1. Draining MachinePool nodes during ASG scale-down.
2. Draining spot instances in MachinePools and MDs (using 2 minutes of warning time to drain the nodes)

AWS node termination handler has support for both cases above. So we can design these together.

[sedefsavas]

Related cluster-api proposal for spot instance termination handling:
kubernetes-sigs/cluster-api#3528

TL;DR
Interruptible label is added to the spot instance nodes by Machine controller by checking AWSMachine.Status.Interruptible field: kubernetes-sigs/cluster-api#3668

Using the interruptible labels on the nodes, we can install termination handler as DaemonSet:
https://github.com/aws/aws-node-termination-handler

[sedefsavas]

Also, once MachinePool Machines proposal is in, when shut-down signal of a node is detected, we can utilize MHC to clean up resources following this proposal: kubernetes-sigs/cluster-api#3528

[sedefsavas]

Validate Amazon FSx for Lustre storage CSI and EFS CSI plugins, similar to existing EBS CSI validation.

[sedefsavas]

Support for AWS EC2 Fleet, which

• allows grouping mixed nodes by specifying different instance types with on-demand and spot purchase models
• launches the lowest price combination of instances to meet the target capacity based on these preferences

[sedefsavas]

Support for AWS Spot Fleet, which by selecting multiple instance types and max price, creates spot instances to meet target capacity.

[richardcase]

@sedefsavas - a great list 👍

[pydctw]

Support cluster autoscaler

[sedefsavas]

AFAIK cluster autoscaler works as is with MachineDeployments.
For MachinePool support, MachinePool Machine proposal must be implemented.
If there is more to it other than adding an e2e test, would be good to have an issue.

[pydctw]

Agreed. I looked into the cluster autoscaler in my previous project and adding annotations on MD will do. This item is more for e2e test.

[richardcase]

And karpenter

[sedefsavas]

Windows nodes support