Skip to content

Commit

Permalink
multi-cluster bootstrap in antctl (antrea-io#3474)
Browse files Browse the repository at this point in the history
Add new subcommands to Create or Delete multi-cluster Resources.

Signed-off-by: hjiajing <hjiajing@vmware.com>
  • Loading branch information
hjiajing committed Jun 8, 2022
1 parent 62a0145 commit 3e1b254
Show file tree
Hide file tree
Showing 13 changed files with 1,433 additions and 4 deletions.
70 changes: 66 additions & 4 deletions docs/multicluster/antctl.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
# Antctl Multi-cluster commands

Starting from version 1.6.0, Antrea supports the `antctl mc` commands, which can
collect information from a leader cluster in a ClusterSet, for troubleshooting
issues in an Antrea Multi-cluster ClusterSet.
collect information from a leader cluster in a ClusterSet for troubleshooting
issues in an Antrea Multi-cluster ClusterSet, create and delete resources in an
Antrea Multi-cluster ClusterSet, and so on. The command `antctl mc get` is supported
since Antrea version 1.6.0 and other commands are supported from 1.7.0. These commands
cannot run inside the `antrea-controller`, `antrea-agent` and `antrea-mc-controller`
Pods. The antctl will look for your kubeconfig file at `$HOME/.kube/config` by default.
You can select a different one by setting the `KUBECONFIG` environment variable or with
`--kubeconfig`.

All antctl Multi-cluster commands can only run correctly after [deploying Antrea
Multi-cluster](./user-guide.md) successfully.
## antctl mc get

- `antctl mc get clusterset` (or `get clustersets`) command can print all
ClusterSets, a specified Clusterset, or the ClusterSet in a specified leader cluster
Expand All @@ -28,3 +33,60 @@ antctl mc get resourceexport [NAME] [-n NAMESPACE] [-clusterid CLUSTERID] [-o js
```

To see the usage examples of these commands, you may also run `antctl mc get [subcommand] --help`.

## antctl mc create

`antctl mc create` command can create access-token and other resources like ClusterSet, ClusterClaims for
Antrea Multi-cluster setup

+ `antctl mc create accesstoken` command can create accesstoken for member clusters.
+ `antctl mc create clusterclaims` command can create two ClusterClaims in a leader or member cluster. One for the leader or member cluster, and another for the ClusterSet.
+ `antctl mc create clusterset` command can create a ClusterSet in a leader or member cluster.

```bash
anctcl mc create accesstoken [NAME] [-n NAMESPACE] [--serviceaccount SERVICE_ACCOUNT] [--role-binding ROLE_BINDING]
antctl mc create clusterclaims [-n NAMESPACE] [--clusterset-id CLUSTERSET_ID] [--cluster-id CLUSTER_ID]
antctl mc create clusterset [NAME] [-n NAMESPACE] [--leader-server LEADER_SERVER] [--service-account SERVICE_ACCOUNT] [--secret SECRET] [--leader-cluster LEADER_CLUSTER_ID]
```

To see the usage examples of these commands, you may also run `antctl mc create [subcommand] --help`.

## antctl mc add

`antctl mc add` command can add a new member cluster to a ClusterSet.

```bash
antctl mc add membercluster [CLUSTER_ID] [-n NAMESPACE] [--clusterset CLUSTERSET] [--service-account SERVICE_ACCOUNT]
```

To see the usage examples of these commands, you may also run `antctl mc add [subcommand] --help`.

## antctl mc delete

`antctl mc delete` command can delete resources in an Antrea Multi-cluster ClusterSet.

+ `antctl mc delete clusterclaims` command can delete the two ClusterClaims in a specified Namespace. One for the leader or member cluster, and another for the ClusterSet.
+ `antctl mc delete clusterset` command can delete a ClusterSet in a leader or member cluster.
+ `antctl mc delete member-cluster` command can delete a member cluster in a specified Antrea Multi-cluster ClusterSet.

```bash
antctl mc delete clusterclaims [-n NAMESPACE]
antctl mc delete clusterset [NAME] [-n NAMESPACE]
antctl mc delete membercluster [MEMBER_CLUSTER_ID] [-n NAMESPACE] [--clusterset CLUSTERSET]
```

To see the usage examples of these commands, you may also run `antctl mc delete [subcommand] --help`.

## antctl mc deploy

`antctl mc deploy` command can deploy Antrea Multi-cluster Controller to a leader or member cluster.

+ `antctl mc deploy leadercluster` command can deploy Antrea Multi-cluster Controller to a leader cluster, and define all the CRDs the leader cluster needed.
+ `antctl mc deploy membercluster` command can deploy Antrea Multi-cluster Controller to a member cluster, and define all the CRDs the member cluster needed.

```bash
antctl mc deploy leadercluster [--antrea-version ANTREA_VERSION] [-n NAMESPACE] [-f PATH_TO_MANIFEST]
antctl mc deploy membercluster [--antrea-version ANTREA_VERSION] [-n NAMESPACE] [-f PATH_TO_MANIFEST]
```

To see the usage examples of these commands, you may also run `antctl mc deploy [subcommand] --help`.
24 changes: 24 additions & 0 deletions pkg/antctl/antctl.go
Original file line number Diff line number Diff line change
Expand Up @@ -565,6 +565,30 @@ var CommandList = &commandList{
supportController: false,
commandGroup: mc,
},
{
cobraCommand: multicluster.AddCmd,
supportAgent: false,
supportController: false,
commandGroup: mc,
},
{
cobraCommand: multicluster.CreateCmd,
supportAgent: false,
supportController: false,
commandGroup: mc,
},
{
cobraCommand: multicluster.DeleteCmd,
supportAgent: false,
supportController: false,
commandGroup: mc,
},
{
cobraCommand: multicluster.DeployCmd,
supportAgent: false,
supportController: false,
commandGroup: mc,
},
},
codec: scheme.Codecs,
}
Expand Down
115 changes: 115 additions & 0 deletions pkg/antctl/raw/multicluster/add/member_cluster.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
// Copyright 2022 Antrea Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package add

import (
"context"
"fmt"
"strings"

"github.com/spf13/cobra"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/rest"
"sigs.k8s.io/controller-runtime/pkg/client"

multiclusterv1alpha1 "antrea.io/antrea/multicluster/apis/multicluster/v1alpha1"
"antrea.io/antrea/pkg/antctl/raw"
multiclusterscheme "antrea.io/antrea/pkg/antctl/raw/multicluster/scheme"
)

type memberClusterOptions struct {
namespace string
clusterSet string
serviceAccount string
}

var memberClusterOpt *memberClusterOptions

var memberClusterExamples = strings.Trim(`
# Add a new member cluster to a ClusterSet
$ antctl mc add membercluster <CLUSTER_ID> -n <NAMESPACE> --clusterset <CLUSTERSET_ID> --service-account <SERVICE_ACCOUNT>
`, "\n")

func (o *memberClusterOptions) validateAndComplete() error {
if o.namespace == "" {
return fmt.Errorf("the Namespace cannot be empty")
}
if o.clusterSet == "" {
return fmt.Errorf("the ClusterSet cannot be empty")
}
if o.serviceAccount == "" {
return fmt.Errorf("the ServiceAccount cannot be empty")
}

return nil
}

func NewMemberClusterCmd() *cobra.Command {
command := &cobra.Command{
Use: "membercluster",
Args: cobra.MaximumNArgs(1),
Short: "Add a new member cluster to a ClusterSet",
Long: "Add a new member cluster to a ClusterSet",
Example: memberClusterExamples,
RunE: memberClusterRunE,
}

o := &memberClusterOptions{}
memberClusterOpt = o
command.Flags().StringVarP(&o.namespace, "namespace", "n", "", "Namespace of member cluster")
command.Flags().StringVarP(&o.clusterSet, "clusterset", "", "", "The name of target ClusterSet to add a new member cluster")
command.Flags().StringVarP(&o.serviceAccount, "service-account", "", "", "ServiceAccount of the member cluster")

return command
}

func memberClusterRunE(cmd *cobra.Command, args []string) error {
if err := memberClusterOpt.validateAndComplete(); err != nil {
return err
}
if len(args) != 1 {
return fmt.Errorf("exactly one NAME is required, got %d", len(args))
}

kubeconfig, err := raw.ResolveKubeconfig(cmd)
if err != nil {
return err
}
restconfigTmpl := rest.CopyConfig(kubeconfig)
raw.SetupKubeconfig(restconfigTmpl)

k8sClient, err := client.New(kubeconfig, client.Options{Scheme: multiclusterscheme.Scheme})
if err != nil {
return err
}

memberClusterID := args[0]
clusterSet := &multiclusterv1alpha1.ClusterSet{}
if err := k8sClient.Get(context.TODO(), types.NamespacedName{Name: memberClusterOpt.clusterSet, Namespace: memberClusterOpt.namespace}, clusterSet); err != nil {
return err
}
for _, member := range clusterSet.Spec.Members {
if member.ClusterID == memberClusterID {
return fmt.Errorf(`the member cluster "%s" was already added to the ClusterSet "%s"`, memberClusterID, memberClusterOpt.clusterSet)
}
}
clusterSet.Spec.Members = append(clusterSet.Spec.Members, multiclusterv1alpha1.MemberCluster{ClusterID: memberClusterID, ServiceAccount: memberClusterOpt.serviceAccount})
if err := k8sClient.Update(context.TODO(), clusterSet); err != nil {
return err
}

fmt.Fprintf(cmd.OutOrStdout(), "The member cluster \"%s\" is added to the ClusterSet \"%s\" successfully\n", memberClusterID, memberClusterOpt.clusterSet)
return nil
}
33 changes: 33 additions & 0 deletions pkg/antctl/raw/multicluster/commands.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ package multicluster
import (
"github.com/spf13/cobra"

"antrea.io/antrea/pkg/antctl/raw/multicluster/add"
"antrea.io/antrea/pkg/antctl/raw/multicluster/create"
deleteCmd "antrea.io/antrea/pkg/antctl/raw/multicluster/delete"
"antrea.io/antrea/pkg/antctl/raw/multicluster/deploy"
"antrea.io/antrea/pkg/antctl/raw/multicluster/get"
)

Expand All @@ -25,8 +29,37 @@ var GetCmd = &cobra.Command{
Short: "Display one or many resources in a ClusterSet",
}

var CreateCmd = &cobra.Command{
Use: "create",
Short: "Create multi-cluster resources",
}

var AddCmd = &cobra.Command{
Use: "add",
Short: "Add a new member cluster to a ClusterSet",
}

var DeleteCmd = &cobra.Command{
Use: "delete",
Short: "Delete multi-cluster resources",
}

var DeployCmd = &cobra.Command{
Use: "deploy",
Short: "Deploy Antrea Multi-cluster Controller to a leader or member cluster",
}

func init() {
GetCmd.AddCommand(get.NewClusterSetCommand())
GetCmd.AddCommand(get.NewResourceImportCommand())
GetCmd.AddCommand(get.NewResourceExportCommand())
CreateCmd.AddCommand(create.NewClusterClaimCmd())
CreateCmd.AddCommand(create.NewAccessTokenCmd())
CreateCmd.AddCommand(create.NewClusterSetCmd())
DeleteCmd.AddCommand(deleteCmd.NewMemberClusterCmd())
DeleteCmd.AddCommand(deleteCmd.NewClusterSetCmd())
DeleteCmd.AddCommand(deleteCmd.NewClusterClaimCmd())
AddCmd.AddCommand(add.NewMemberClusterCmd())
DeployCmd.AddCommand(deploy.NewLeaderClusterCmd())
DeployCmd.AddCommand(deploy.NewMemberClusterCmd())
}
Loading

0 comments on commit 3e1b254

Please sign in to comment.