diff --git a/docs/antrea-ipam.md b/docs/antrea-ipam.md index 7bb6cbac774..8b96136d4f9 100644 --- a/docs/antrea-ipam.md +++ b/docs/antrea-ipam.md @@ -24,7 +24,6 @@ * [CNI IPAM configuration](#cni-ipam-configuration) * [Configuration with `NetworkAttachmentDefinition` CRD](#configuration-with-networkattachmentdefinition-crd) * [`IPPool` CRD](#ippool-crd) - * [Secondary Network creation with Multus](#secondary-network-creation-with-multus) ## Running NodeIPAM within Antrea Controller @@ -321,10 +320,9 @@ will get same IP after recreated. ## IPAM for Secondary Network -With the AntreaIPAM feature, Antrea can allocate IPs for Pod secondary networks. At the -moment, AntreaIPAM supports secondary networks managed by [Multus](https://github.com/k8snetworkplumbingwg/multus-cni), -we will add support for [secondary networks managed by Antrea](feature-gates.md#secondarynetwork) -in the future. +With the AntreaIPAM feature, Antrea can allocate IPs for Pod secondary networks, +including both [secondary networks managed by Antrea](secondary-network.md) and +secondary networks managed by [Multus](cookbooks/multus). ### Prerequisites @@ -470,13 +468,40 @@ spec: prefixLength: 64 ``` -VLAN ID in the IP range subnet definition of `IPPool` CRD is not supported for -secondary network IPAM. +When used for Antrea secondary VLAN network, the VLAN set in an `IPPool` IP +range will be passed to the VLAN interface configuration. For example: -### Secondary Network creation with Multus +```yaml +apiVersion: "crd.antrea.io/v1alpha2" +kind: IPPool +metadata: + name: ipv4-pool-1 +spec: + ipVersion: 4 + ipRanges: + - cidr: "10.10.1.0/26" + gateway: "10.10.1.1" + prefixLength: 24 + vlan: 100 + +--- +apiVersion: "k8s.cni.cncf.io/v1" +kind: NetworkAttachmentDefinition +metadata: + name: ipv4-net-1 +spec: + { + "cniVersion": "0.3.0", + "type": "antrea", + "networkType": "vlan", + "ipam": { + "type": "antrea", + "ippools": [ "ipv4-pool-1" ] + } + } +``` + +You can refer to the [Antrea secondary network document](secondary-network.md) +for more information about Antrea secondary VLAN network configuration. -To leverage Antrea for secondary network IPAM, Antrea must be used as the CNI -for the Pods' primary network, while the secondary networks are implemented by -other CNIs which are managed by Multus. The [Antrea + Multus guide](cookbooks/multus) -talks about how to use Antrea with Multus, including the option of using Antrea -IPAM for secondary networks. +For other network types, the VLAN field in the `IPPool` will be ignored. diff --git a/docs/secondary-network.md b/docs/secondary-network.md new file mode 100644 index 00000000000..d02f6f30c68 --- /dev/null +++ b/docs/secondary-network.md @@ -0,0 +1,142 @@ +# Antrea Secondary Network Support + +Antrea can work with Multus, in which case Antrea is the primary CNI of the +Kubernetes cluster and provisions the "primary" network interfaces of Pods; +while Multus manages secondary networks and executes other CNIs to create +secondary network interfaces of Pods. The [Antrea + Multus guide](cookbooks/multus) +talks about how to use Antrea with Multus. + +Starting with Antrea v1.15, Antrea can also provision secondary network +interfaces and connect them to VLAN networks. This document describes Antrea's +native support for VLAN secondary networks. + +## Prerequisites + +Native secondary network support is still an alpha feature and is disabled by +default. To use the feature, the `SecondaryNetwork` feature gate must be enabled +in the `antrea-agent` configuration. If you need IPAM for the secondary +interfaces, you should also enable the `AntreaIPAM` feature gate in both +`antrea-agent` and `antrea-controller` configuration. At the moment, Antrea IPAM +is the only available IPAM option for secondary networks managed by Antrea. The +`antrea-config` ConfigMap with the two feature gates enables is like the +following: + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: antrea-config + namespace: kube-system +data: + antrea-controller.conf: | + featureGates: + AntreaIPAM: true + antrea-agent.conf: | + featureGates: + AntreaIPAM: true + SecondaryNetwork: true +``` + +Antrea leverages the `NetworkAttachmentDefinition` CRD from [Kubernetes Network +Plumbing Working Group](https://github.com/k8snetworkplumbingwg/multi-net-spec) +to define secondary networks. You can import the CRD to your cluster using the +following command: + +```bash +kubectl apply -f https://github.com/k8snetworkplumbingwg/network-attachment-definition-client/raw/master/artifacts/networks-crd.yaml +``` + +## Secondary OVS bridge configuration + +A VLAN secondary interface will be connected to a separate OVS bridge on the +Node. You can specify the secondary OVS bridge configuration in the +`antrea-agent` configuration, and `antrea-agent` will automatically create the +OVS bridge based on the configuration. For example, the following configuration +will create an OVS bridge named `br-secondary`, with a physical interface +`eth1`. + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: antrea-config + namespace: kube-system +data: + antrea-agent.conf: | + secondaryNetwork: + ovsBridges: [{"bridgeName": "br-secondary", "physicalInterfaces": ["eth1"]}] +``` + +At the moment, Antrea supports only a single OVS bridge for secondary networks, +and supports only a single physical interface on the bridge. The physical +interface cannot be the Node's management interface, otherwise a Node's +management network connectivity can be broken after `antrea-agent` creates the +OVS bridge and moves the management interface to the bridge. + +## Secondary VLAN network configuration + +A secondary VLAN network is defined by a NetworkAttachmentDefinition CR. For +example, the following NetworkAttachmentDefinition defines a VLAN network +`vlan100`. + +```yaml +apiVersion: "k8s.cni.cncf.io/v1" +kind: NetworkAttachmentDefinition +metadata: + name: vlan100 +spec: + config: '{ + "cniVersion": "0.3.0", + "type": "antrea", + "networkType": "vlan", + "mtu": 1500, + "vlan": 100, + "ipam": { + "type": "antrea", + "ippools": ["vlan100-ipv4", "vlan100-ipv6"] + } + }' +``` + +`antrea-agent` will connect Pod secondary interfaces belonging to a VLAN network +to the secondary OVS bridge on the Node. If a non-zero VLAN is speficied in the +network's `config`, `antrea-agent` will configure the VLAN ID on the OVS port, +so the interface's traffic will be isolated within the VLAN. And before the +traffic is forwarded out the Node via the secondary bridge's physical interface, +OVS will insert the VLAN tag in the packets. + +A few extra notes about the NetworkAttachmentDefinition `config` fields: + +* `type` - must be set to `antrea`. +* `networkType` - the only supported network type is `vlan` as of now. +* `mtu` - defaults to 1500 if not set. +* `vlan` - can be set to 0 or a valid VLAN ID (1 - 4094). Defaults to 0. The +VLAN ID can also be specified as part of the spec of an IPPool referenced in the +`ipam` section, but `vlan` in NetworkAttachmentDefinition `config` will override +the VLAN in IPPool(s) if both are set. +* `ipam` - it is optional. If not set, the secondary interfaces created for the +network won't have an IP address allocated. For more information about secondary +network IPAM configuration, please refer to the [Antrea IPAM document](antrea-ipam.md#ipam-for-secondary-network). + +## Pod secondary interface configuration + +You can create a Pod with secondary network interfaces by adding the +`k8s.v1.cni.cncf.io/networks` annotation to the Pod. The following example Pod +includes two secondary interfaces, one in network `vlan100` which should be +created in the same Namespace as the Pod, the other in network `vlan200` which +is created in Namespace `networks`. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: sample-pod + labels: + app: antrea-secondary-network-demo + annotations: + k8s.v1.cni.cncf.io/networks: [vlan100, {name: vlan200, namespace: networks, interface: eth200"] +spec: + containers: + - name: toolbox + image: antrea/toolbox:latest +``` diff --git a/hack/.notableofcontents b/hack/.notableofcontents index c895665f8b7..476abf7da99 100644 --- a/hack/.notableofcontents +++ b/hack/.notableofcontents @@ -39,5 +39,6 @@ docs/octant-plugin-installation.md docs/os-issues.md docs/ovs-offload.md docs/prometheus-integration.md +docs/secondary-network.md docs/security.md docs/traffic-encryption.md