Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use same MTU as uplink for bridge port #6577

Merged
merged 2 commits into from
Aug 6, 2024
Merged

Use same MTU as uplink for bridge port #6577

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0182d3f
Use same MTU as uplink for bridge port
antoninbas Jul 31, 2024
7c844cc
Address review comments
antoninbas Aug 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build/charts/antrea/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ Kubernetes: `>= 1.19.0-0`
| controller.selfSignedCert | bool | `true` | Indicates whether to use auto-generated self-signed TLS certificates. If false, a Secret named "antrea-controller-tls" must be provided with the following keys: ca.crt, tls.crt, tls.key. |
| controller.tolerations | list | `[{"key":"CriticalAddonsOnly","operator":"Exists"},{"effect":"NoSchedule","key":"node-role.kubernetes.io/master"},{"effect":"NoSchedule","key":"node-role.kubernetes.io/control-plane"},{"effect":"NoExecute","key":"node.kubernetes.io/unreachable","operator":"Exists","tolerationSeconds":0}]` | Tolerations for the antrea-controller Pod. |
| controllerImage | object | `{"pullPolicy":"IfNotPresent","repository":"antrea/antrea-controller-ubuntu","tag":""}` | Container image to use for the antrea-controller component. |
| defaultMTU | int | `0` | Default MTU to use for the host gateway interface and the network interface of each Pod. By default, antrea-agent will discover the MTU of the Node's primary interface and adjust it to accommodate for tunnel encapsulation overhead if applicable. |
| defaultMTU | int | `0` | Default MTU to use for the host gateway interface and the network interface of each Pod. By default, antrea-agent will discover the MTU of the Node's primary interface and adjust it to accommodate for tunnel encapsulation overhead if applicable. If the MTU is updated, the new value will only be applied to new workloads. |
| disableTXChecksumOffload | bool | `false` | Disable TX checksum offloading for container network interfaces. It's supposed to be set to true when the datapath doesn't support TX checksum offloading, which causes packets to be dropped due to bad checksum. It affects Pods running on Linux Nodes only. |
| dnsServerOverride | string | `""` | Address of DNS server, to override the kube-dns Service. It's used to resolve hostnames in a FQDN policy. |
| egress.exceptCIDRs | list | `[]` | CIDR ranges to which outbound Pod traffic will not be SNAT'd by Egresses. |
Expand Down
1 change: 1 addition & 0 deletions build/charts/antrea/conf/antrea-agent.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@ disableTXChecksumOffload: {{ .Values.disableTXChecksumOffload }}
# Default MTU to use for the host gateway interface and the network interface of each Pod.
# If omitted, antrea-agent will discover the MTU of the Node's primary interface and
# also adjust MTU to accommodate for tunnel encapsulation overhead (if applicable).
# If the MTU is updated, the new value will only be applied to new workloads.
defaultMTU: {{ .Values.defaultMTU }}

# packetInRate defines the OVS controller packet rate limits for different
Expand Down
3 changes: 2 additions & 1 deletion build/charts/antrea/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,8 @@ multicast:
# -- Default MTU to use for the host gateway interface and the network interface
# of each Pod. By default, antrea-agent will discover the MTU of the Node's
# primary interface and adjust it to accommodate for tunnel encapsulation
# overhead if applicable.
# overhead if applicable. If the MTU is updated, the new value will only be
# applied to new workloads.
defaultMTU: 0

# -- packetInRate defines the OVS controller packet rate limits for different
Expand Down
5 changes: 3 additions & 2 deletions build/yamls/antrea-aks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3891,6 +3891,7 @@ data:
# Default MTU to use for the host gateway interface and the network interface of each Pod.
# If omitted, antrea-agent will discover the MTU of the Node's primary interface and
# also adjust MTU to accommodate for tunnel encapsulation overhead (if applicable).
# If the MTU is updated, the new value will only be applied to new workloads.
defaultMTU: 0

# packetInRate defines the OVS controller packet rate limits for different
Expand Down Expand Up @@ -5125,7 +5126,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: cce7d6644fb552607ebeda9bf30a5fafa871dd4382afc609500fcb493b61768c
checksum/config: f950d38c3e5f05b4e6290aae92fc46eeda9126a68a0ed6b88eee7f5c4c6fb491
labels:
app: antrea
component: antrea-agent
Expand Down Expand Up @@ -5363,7 +5364,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: cce7d6644fb552607ebeda9bf30a5fafa871dd4382afc609500fcb493b61768c
checksum/config: f950d38c3e5f05b4e6290aae92fc46eeda9126a68a0ed6b88eee7f5c4c6fb491
labels:
app: antrea
component: antrea-controller
Expand Down
5 changes: 3 additions & 2 deletions build/yamls/antrea-eks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3891,6 +3891,7 @@ data:
# Default MTU to use for the host gateway interface and the network interface of each Pod.
# If omitted, antrea-agent will discover the MTU of the Node's primary interface and
# also adjust MTU to accommodate for tunnel encapsulation overhead (if applicable).
# If the MTU is updated, the new value will only be applied to new workloads.
defaultMTU: 0

# packetInRate defines the OVS controller packet rate limits for different
Expand Down Expand Up @@ -5125,7 +5126,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: cce7d6644fb552607ebeda9bf30a5fafa871dd4382afc609500fcb493b61768c
checksum/config: f950d38c3e5f05b4e6290aae92fc46eeda9126a68a0ed6b88eee7f5c4c6fb491
labels:
app: antrea
component: antrea-agent
Expand Down Expand Up @@ -5364,7 +5365,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: cce7d6644fb552607ebeda9bf30a5fafa871dd4382afc609500fcb493b61768c
checksum/config: f950d38c3e5f05b4e6290aae92fc46eeda9126a68a0ed6b88eee7f5c4c6fb491
labels:
app: antrea
component: antrea-controller
Expand Down
5 changes: 3 additions & 2 deletions build/yamls/antrea-gke.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3891,6 +3891,7 @@ data:
# Default MTU to use for the host gateway interface and the network interface of each Pod.
# If omitted, antrea-agent will discover the MTU of the Node's primary interface and
# also adjust MTU to accommodate for tunnel encapsulation overhead (if applicable).
# If the MTU is updated, the new value will only be applied to new workloads.
defaultMTU: 0

# packetInRate defines the OVS controller packet rate limits for different
Expand Down Expand Up @@ -5125,7 +5126,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: e30c52c9fcb04d362d018e846cf72dc633c5e891e02b3ebb87fab4d7ee08e15a
checksum/config: 5bab13c466e83f8a14191bfb9aad49229945c442808ea135f80cafe5e21be5f3
labels:
app: antrea
component: antrea-agent
Expand Down Expand Up @@ -5361,7 +5362,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: e30c52c9fcb04d362d018e846cf72dc633c5e891e02b3ebb87fab4d7ee08e15a
checksum/config: 5bab13c466e83f8a14191bfb9aad49229945c442808ea135f80cafe5e21be5f3
labels:
app: antrea
component: antrea-controller
Expand Down
5 changes: 3 additions & 2 deletions build/yamls/antrea-ipsec.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3904,6 +3904,7 @@ data:
# Default MTU to use for the host gateway interface and the network interface of each Pod.
# If omitted, antrea-agent will discover the MTU of the Node's primary interface and
# also adjust MTU to accommodate for tunnel encapsulation overhead (if applicable).
# If the MTU is updated, the new value will only be applied to new workloads.
defaultMTU: 0

# packetInRate defines the OVS controller packet rate limits for different
Expand Down Expand Up @@ -5138,7 +5139,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 73a49a9a8508cc8fb94eb2c770bb3589e68d9623327231943cba60a48716568a
checksum/config: 7212fbcdde8fe4be00f31ebbbcd7b03a7335666b4af245eed7dac1ba9e99118c
checksum/ipsec-secret: d0eb9c52d0cd4311b6d252a951126bf9bea27ec05590bed8a394f0f792dcb2a4
labels:
app: antrea
Expand Down Expand Up @@ -5420,7 +5421,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 73a49a9a8508cc8fb94eb2c770bb3589e68d9623327231943cba60a48716568a
checksum/config: 7212fbcdde8fe4be00f31ebbbcd7b03a7335666b4af245eed7dac1ba9e99118c
labels:
app: antrea
component: antrea-controller
Expand Down
5 changes: 3 additions & 2 deletions build/yamls/antrea.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3891,6 +3891,7 @@ data:
# Default MTU to use for the host gateway interface and the network interface of each Pod.
# If omitted, antrea-agent will discover the MTU of the Node's primary interface and
# also adjust MTU to accommodate for tunnel encapsulation overhead (if applicable).
# If the MTU is updated, the new value will only be applied to new workloads.
defaultMTU: 0

# packetInRate defines the OVS controller packet rate limits for different
Expand Down Expand Up @@ -5125,7 +5126,7 @@ spec:
kubectl.kubernetes.io/default-container: antrea-agent
# Automatically restart Pods with a RollingUpdate if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 20130c4a5dbfeec75182bc3053288f64c06d0350b34c86675ac88d5961c47853
checksum/config: 616b79b8deedba740ff992ca870b346c64c1dde5e3381436dc2cb24c0bd98ead
labels:
app: antrea
component: antrea-agent
Expand Down Expand Up @@ -5361,7 +5362,7 @@ spec:
annotations:
# Automatically restart Pod if the ConfigMap changes
# See https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
checksum/config: 20130c4a5dbfeec75182bc3053288f64c06d0350b34c86675ac88d5961c47853
checksum/config: 616b79b8deedba740ff992ca870b346c64c1dde5e3381436dc2cb24c0bd98ead
labels:
app: antrea
component: antrea-controller
Expand Down
7 changes: 7 additions & 0 deletions docs/noencap-hybrid-modes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,13 @@ spec:
value: "true"
```

Note that changing the traffic mode in an existing cluster, where Antrea is
currently installed or was previously installed, may require restarting existing
workloads. In particular, the choice of traffic mode has an impact on the MTU
value used for Pod network interfaces. When changing the traffic mode from
`NoEncap` to `Encap`, existing workloads should be restarted, so that new
luolanzone marked this conversation as resolved.
Show resolved Hide resolved
network interfaces with a lower MTU value can be created.

## Hybrid Mode

Let us start from `Hybrid` mode which is simpler to configure. `Hybrid` mode
Expand Down
5 changes: 5 additions & 0 deletions pkg/agent/agent_linux.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,11 +129,16 @@ func (i *Initializer) ConnectUplinkToOVSBridge() error {
externalIDs := map[string]interface{}{
interfacestore.AntreaInterfaceTypeKey: interfacestore.AntreaHost,
}
// We request the same MTU for the bridge interface as for the uplink adapter. If we don't,
// OVS will default to the lowest MTU among all existing bridge ports, including container
// ports. There may be some existing workloads with a lower MTU, and using that lower value
// may impact host connectivity.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we add additional comment that big packet will be fragmented when it passthrough lower MTU interfaces (e.g. old antrea-gw0)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We always update the MTU of antrea-gw0 to the correct value:

antrea/pkg/agent/agent.go

Line 698 in e9990f8

if err := i.setInterfaceMTU(i.hostGateway, i.networkConfig.InterfaceMTU); err != nil {

Do you have another case in mind?

bridgedUplinkName, exists, err := util.PrepareHostInterfaceConnection(
i.ovsBridgeClient,
uplinkNetConfig.Name,
int32(i.nodeConfig.HostInterfaceOFPort),
externalIDs,
i.nodeConfig.NodeTransportInterfaceMTU,
)
if err != nil {
return err
Expand Down
1 change: 1 addition & 0 deletions pkg/agent/secondarynetwork/init_linux.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ func (c *Controller) Initialize() error {
map[string]interface{}{
interfacestore.AntreaInterfaceTypeKey: interfacestore.AntreaHost,
},
0, // do not request a specific MTU
)
if err != nil {
return err
Expand Down
12 changes: 10 additions & 2 deletions pkg/agent/util/net_linux.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -404,14 +404,16 @@ func removeInterfaceAltName(name string, altName string) error {
// PrepareHostInterfaceConnection prepares host interface connection to the OVS bridge client by:
// 1. Renaming the host interface (a bridged suffix will be added to it).
// 2. Creating an internal port (original name of the host interface will be used here).
// 3. Moving IPs of host interface to this new link/internal-port.
// 4. Moving routes of host interface to the new link/internal-port.
// 3. Set the MTU of this new link/internal-port to the provided mtu parameter value, unless mtu is zero.
// 4. Moving IPs of host interface to this new link/internal-port.
// 5. Moving routes of host interface to the new link/internal-port.
// and returns the bridged name, true if it already exists, and error.
func PrepareHostInterfaceConnection(
bridge ovsconfig.OVSBridgeClient,
ifaceName string,
ifaceOFPort int32,
externalIDs map[string]interface{},
mtu int,
) (string, bool, error) {
bridgedName := GenerateUplinkInterfaceName(ifaceName)
// If the port already exists, just return.
Expand Down Expand Up @@ -453,6 +455,12 @@ func PrepareHostInterfaceConnection(
return "", false, fmt.Errorf("failed to set link up: %v", err)
}

if mtu > 0 {
antoninbas marked this conversation as resolved.
Show resolved Hide resolved
if err := bridge.SetInterfaceMTU(ifaceName, mtu); err != nil {
return "", false, fmt.Errorf("failed to set bridge interface MTU: %w", err)
}
}
Comment on lines +458 to +462
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not enforced if the port already exists (as determined by the call to bridge.GetOFPort at the beginning of the function). My take is that this is consistent with how we handle other configurations at the moment (e.g., we don't call SetLinkUp unconditionally).


// Check if interface is configured with an IPv6 address: if it is, we need to ensure that IPv6
// is enabled on the OVS internal port as we need to move all IP addresses over.
for _, ip := range ifaceIPs {
Expand Down