Release v0.12.3

Fixed

• It was discovered that the AntreaProxy implementation has an upper-bound for the number of Endpoints it can support for each Service: we increase this upper-bound from ~500 to 800, log a warning for Services with a number of Endpoints greater than 800, and arbitrarily drop some Endpoints so we can still provide load-balancing for the Service. (#2101, @hongliangl)
• Fix Antrea-native policy with multiple AppliedTo selectors: some rules were never realized by the Agents as they thought they had only received partial information from the Controller. (#2084, @tnqn)
• Fix re-installation of the OpenFlow groups when the OVS daemons are restarted to ensure that AntreaProxy keeps functioning. (#2134, @antoninbas)
• Fix audit logging on Windows Nodes: the log directory was not configured properly, causing Agent initialization to fail on Windows when the AntreaPolicy feature was enabled. (#2052, @antoninbas) [Windows]
• Use correct output format for CNI Add in networkPolicyOnly mode: this was not an issue with Docker but was causing failures with containerd. (#2037, @antoninbas @dantingl)
• Fix audit logging of IPv6 traffic for Antrea-native policies: IPv6 packets were ignored by the Agent instead of being parsed and logged to file. (#1990, @antoninbas)
• Fix Status updates for ClusterNetworkPolicies. (#2036, @Dyanngg)

Release v1.0.1

Fixed

• It was discovered that the AntreaProxy implementation has an upper-bound for the number of Endpoints it can support for each Service: we increase this upper-bound from ~500 to 800, log a warning for Services with a number of Endpoints greater than 800, and arbitrarily drop some Endpoints so we can still provide load-balancing for the Service. (#2101, @hongliangl)
• Fix Antrea-native policy with multiple AppliedTo selectors: some rules were never realized by the Agents as they thought they had only received partial information from the Controller. (#2084, @tnqn)
• Fix re-installation of the OpenFlow groups when the OVS daemons are restarted to ensure that AntreaProxy keeps functioning. (#2134, @antoninbas)
• Fix IPFIX flow records exported by the Antrea Agent. (#2089, @zyiou)
  • If a connection spanned multiple export cycles, it wasn't handled properly and no record was sent for the connection
  • If a connection spanned a single export cycle, a single record was sent but "delta counters" were set to 0 which caused flow visualization to omit the flow in dashboards
• Fix incorrect stats reporting for ingress rules of some NetworkPolicies: some types of traffic were bypassing the OVS table keeping track of statistics once the connection was established, causing packet and byte stats to be incorrect. (#2078, @ceclinux)
• Fix the retry logic when enabling the OVS bridge local interface on Windows Nodes. (#2081, @antoninbas) [Windows]

Release v0.13.2

Fixed

• It was discovered that the AntreaProxy implementation has an upper-bound for the number of Endpoints it can support for each Service: we increase this upper-bound from ~500 to 800, log a warning for Services with a number of Endpoints greater than 800, and arbitrarily drop some Endpoints so we can still provide load-balancing for the Service. (#2101, @hongliangl)
• Fix Antrea-native policy with multiple AppliedTo selectors: some rules were never realized by the Agents as they thought they had only received partial information from the Controller. (#2084, @tnqn)
• Fix re-installation of the OpenFlow groups when the OVS daemons are restarted to ensure that AntreaProxy keeps functioning. (#2134, @antoninbas)
• Fix the retry logic when enabling the OVS bridge local interface on Windows Nodes. (#2081, @antoninbas) [Windows]
• Fix audit logging on Windows Nodes: the log directory was not configured properly, causing Agent initialization to fail on Windows when the AntreaPolicy feature was enabled. (#2052, @antoninbas) [Windows]
• When selecting the Pods corresponding to a Service for which NodePortLocal has been enabled, Pods should be filtered by Namespace. (#1927, @chauhanshubham)
• Correctly handle Service Type changes for NodePortLocal, and update Pod annotations accordingly. (#1936, @chauhanshubham)
• Use correct output format for CNI Add in networkPolicyOnly mode: this was not an issue with Docker but was causing failures with containerd. (#2037, @antoninbas @dantingl)
• Fix audit logging of IPv6 traffic for Antrea-native policies: IPv6 packets were ignored by the Agent instead of being parsed and logged to file. (#1990, @antoninbas)
• Fix Status updates for ClusterNetworkPolicies. (#2036, @Dyanngg)

Release v1.0.0

Includes all the changes from [0.13.1].

The AntreaPolicy feature is graduated from Alpha to Beta and is therefore enabled by default.

Added

• Add [Egress] feature to configure SNAT policies for Pod-to-external traffic. [Alpha - Feature Gate: Egress]
  • A new Egress CRD is introduced to define SNAT policies (#1433, [@jianjuns])
  • Update the datapath to implement Egress: on Windows Nodes, everything is implemented in OVS, while on Linux Nodes, OVS marks packets and sends them to the host network namespace, where iptables handles SNAT (#1892 #1969 #1998, [@jianjuns], [@tnqn])
  • A new EgressGroup control plane API is introduced: the Controller computes group membership for each policy and sends this information to the Agents (#1965, [@tnqn])
  • Implement the EgressGroup control plane API in the Agent (#2026, [@tnqn] [@ceclinux])
  • Document the Egress feature and its datapath implementation (#2041 #2044, [@jianjuns] [@tnqn])
• Add support for the "Reject" action in Antrea-native policies as an alternative to "Drop" (which silently drops packets). (#1888, [@GraysonWu])
  • For rejected TCP connections, the Agent will send a TCP RST packet
  • For UDP and SCTP, the Agent will send an ICMP message with Type 3 (Destination Unreachable) and Code 10 (Host administratively prohibited)
• Add support for nesting in the [ClusterGroup CRD]: a ClusterGroup can now reference a list of ClusterGroups, but only one level of nesting is supported. (#1920, [@Dyanngg])
• Add ability to specify multiple IPBlocks when defining a ClusterGroup. (#1993, [@Dyanngg])
• Support for IPv6 (IPv6-only and dual-stack clusters) in the FlowAggregator and in the reference ELK stack. (#1819 #1962, [@dreamtalen])
• Add support for arm/v7 and arm64 to the main Antrea Docker image for Linux (antrea/antrea-ubuntu) instead of using a separate image. (#1994, [@antoninbas])
• Add support for live-traffic tracing in Traceflow: rather than injecting a Traceflow packet, we can monitor real traffic and update the Traceflow Status when a matching packet is observed. (#2005 #2029, [@jianjuns])
  • The captured packet is reported as part of the Traceflow request Status
  • Live-traffic tracing supports a "Dropped-Only" filter which will only capture packets dropped by the datapath
• Introduce a new optional mutating webhook to automatically label all Namespaces and Services with their name (antrea.io/metadata.name: <resourceName>); this allows NetworkPolicies and ClusterGroup to easily select these resources by name. (#1690, [@abhiraut] [@Dyanngg])
• Add support for rule-level statistics for Antrea-native policies, when the NetworkPolicyStats feature is enabled: rules are identified by their name, which can be user-provided or auto-generated. (#1780, [@ceclinux])
• Add TCP connection state information to the IPFIX records sent by the FlowExporter, and improve handling of "dying" connections. (#1904, [@zyiou])
• Add information about the flow type (intra-Node, inter-Node, Pod-to-external) to the IPFIX records sent by the FlowExporter. (#2000, [@dreamtalen])
• Add support for dumping OVS flows related to a Service with the "antctl get of" command. (#1877, [@jianjuns])
• Randomly generate a cluster UUID in the Antrea Controller and make it persistent by storing it to a ConfigMap ("antrea-cluster-identity"). (#1805, [@antoninbas])
• Add support for IPv6 to "antctl traceflow". (#1995, [@luolanzone])

Changed

• Rename all Antrea API groups from *.antrea.tanzu.vmware.com to *.antrea.io. (#1799, [@hongliangl])
  • All legacy groups will be supported until December 2021
  • See the [API documentation] for more details and information on how to upgrade client applications which use the Antrea API (#2031, [@antoninbas])
• Change the export mechanism for the FlowExporter in the Antrea Agent: instead of exporting all flows periodically with a fixed interval, we introduce an "active timeout" and an "idle timeout", and flow information is exported differently based on flow activity. (#1714, [@srikartati])
• Add rate-limiting in the Agent for PacketIn messages sent by the OVS datapath: this can help limit the CPU usage when too many messages are sent by OVS. (#2015, [@GraysonWu])
• Output partial result when a Traceflow request initiated by antctl fails or times out, as it can still provide useful information. (#1879, [@jianjuns])
• Ensure that "antctl version" always outputs the client version, even when antctl cannot connect to the Antrea apiserver. (#1876, [@antoninbas])
• Extract the group member calculation for the NetworkPolicy implementation in the Controller to its own module, so it can be reused for different features which need to calculate groups of endpoints based on a given selection criteria; p
  erformance (CPU and memory usage) is also improved. (#1937, [@tnqn])
• Optimize the computation of unions of sets when processing NetworkPolicies in the Controller. (#1938, [@tnqn])
• Optimize the computation of symmetric differences of sets in the Agent (NodePortLocal) and in the Controller (NetworkPolicy processing). (#1944, [@tnqn])
• Move mutable ConfigMap resources out of the deployment YAML and create them programmatically instead; this facilitates integration with other projects such as kapp. (#1983, [@hty690])
• Improve error logs when the Antrea Agent's connection to the Controller times out, and introduce a dedicated health check in the Agent to report the connection status. (#1946, [@hty690])
• Support user-provided signed OVS binaries in Windows installation script. (#1963, [@lzhecheng]) [Windows]
• When NodePortLocal is enabled on a Pod, do not allocate new ports on the host for Pod containers with HostPort enabled. (#2024, [@annakhm])
• Use "distroless" Docker image for the FlowAggregator to reduce its size. (#2004 #2016, [@hanlins] [@dreamtalen])
• Improve reference Kibana dashboards for flow visualization and update the documentation for flow visualization with more up-to-date Kibana screenshots. (#1933, [@zyiou])
• Reject unsupported positional arguments in antctl commands. (#2011, [@hty690])
• Reduce log verbosity for PacketIn messages received by the Agent. (#2046, [@jianjuns])
• Improve Windows documentation to cover running Antrea as a Windows service, which is required when using containerd as the container runtime. (#1874, [@lzhecheng] [@jayunit100]) [Windows]
• Update the documentation for hardware offload support. (#1943, [@Mmduh-483])
• Document IPv6 support for Traceflow. (#1996, [@gran-vmv])
• Remove old references to Ubuntu 18.04 from the documentation. (#1960, [@shadowlan])

Fixed

• Fix audit logging on Windows Nodes: the log directory was not configured properly, causing Agent initialization to fail on Windows when the AntreaPolicy feature was enabled. (#2052, [@antoninbas]) [Windows]
• When selecting the Pods corresponding to a Service for which NodePortLocal has been enabled, Pods should be filtered by Namespace. (#1927, [@chauhanshubham])
• Correctly handle Service Type changes for NodePortLocal, and update Pod annotations accordingly. (#1936, [@chauhanshubham])
• Use correct output format for CNI Add in networkPolicyOnly mode: this was not an issue with Docker but was causing failures with containerd. (#2037, [@antoninbas] [@dantingl])
• Fix ...

Release v0.13.1

Fixed

• Clean up stale IP addresses on Antrea host gateway interface. (#1900, @antoninbas)
  • If a Node leaves and later rejoins a cluster, a new Pod CIDR may be allocated to the Node for each supported IP family and the gateway receives a new IP address (first address in the CIDR)
  • If the previous addresses are not removed from the gateway, we observe connectivity issues across Nodes
• Update libOpenflow to avoid crash in Antrea Agent for certain Traceflow requests. (#1833, @antoninbas)
• Fix the deletion of stale port forwarding iptables rules installed for NodePortLocal, occurring when the Antrea Agent restarts. (#1887, @monotosh-avi)
• Fix output formatting for the "antctl trace-packet" command: the result was displayed as a Go struct variable and newline characters were not rendered, making it hard to read. (#1897, @jianjuns)

Release v0.12.2

Fixed

• Ensure that NodePort traffic does not bypass NetworkPolicies. (#1816, @tnqn)
  • NodePort traffic for which ExternalTrafficPolicy is set to Cluster goes through SNAT before NetworkPolicies are enforced; after SNAT the source IP is the IP of the local gateway interface (antrea-gw0)
  • Users will need to define the appropriate NetworkPolicies to allow ingress access to isolated Pods for NodePort traffic
  • This new behavior only applies to Linux Nodes using the OVS system datapath (default)
• Clean up stale IP addresses on Antrea host gateway interface. (#1900, @antoninbas)
  • If a Node leaves and later rejoins a cluster, a new Pod CIDR may be allocated to the Node for each supported IP family and the gateway receives a new IP address (first address in the CIDR)
  • If the previous addresses are not removed from the gateway, we observe connectivity issues across Nodes

Release v0.11.3

Fixed

• Ensure that NodePort traffic does not bypass NetworkPolicies. (#1816, @tnqn)
  • NodePort traffic for which ExternalTrafficPolicy is set to Cluster goes through SNAT before NetworkPolicies are enforced; after SNAT the source IP is the IP of the local gateway interface (antrea-gw0)
  • Users will need to define the appropriate NetworkPolicies to allow ingress access to isolated Pods for NodePort traffic
  • This new behavior only applies to Linux Nodes using the OVS system datapath (default)
• Clean up stale IP addresses on Antrea host gateway interface. (#1900, @antoninbas)
  • If a Node leaves and later rejoins a cluster, a new Pod CIDR may be allocated to the Node for each supported IP family and the gateway receives a new IP address (first address in the CIDR)
  • If the previous addresses are not removed from the gateway, we observe connectivity issues across Nodes

Release v0.13.0

Includes all the changes from [0.12.1].

Added

• Add [NodePortLocal] feature to improve integration with external load-balancers. (#1459 #1743 #1758, [@monotosh-avi] [@shubhamavi] [@hemantavi]) [Alpha - Feature Gate: NodePortLocal]
  • Services can be annotated with "nodeportlocal.antrea.io/enabled" to indicate that NodePortLocal should be enabled for this Service's Pod Endpoints
  • For each container port exposed by such a Pod, the Antrea Agent will allocate a local Node port value and traffic sent to this Node port will be forwarded to the container port using DNAT
  • The mapping from allocated Node ports to container ports is stored in a new Pod annotation, "nodeportlocal.antrea.io", e.g. to be consumed by external load-balancers
• Introduce the [ClusterGroup CRD] to logically group different network endpoints and reference them together in Antrea-native policies. (#1782, [@abhiraut] [@Dyanngg])
  • The extra level of indirection enables separation between workload selection and policy definition
  • ClusterGroups can be referenced in Antrea ClusterNetworkPolicies, either in the AppliedTo or as peers in policy rules (#1750 #1734)
  • In addition to the Pod / Namespace selectors and ipBlocks, ClusterGroups can reference a Service by name directly, and all Pod Endpoints for this Service will be included in the ClusterGroup (#1797)
  • ClusterGroups can also select ExternalEntitites, which are used to represent labelled non-Pod endpoints (#1828)
  • The ClusterGroup CRD includes a Status subresource used to indicate whether the Antrea Controller has already computed the membership list for the group (#1778)
  • New APIs are defined in "controlplane.antrea.tanzu.vmware.com/v1beta2": "/clustergroupmembers" retrieves the list of members of a group and "/groupassociations" retrieves the list of groups that a given endpoint (Pod or ExternalEntity) belongs to (#1688)
• Add support for containerd runtime on Windows Nodes. (#1781 #1832, [@ruicao93]) [Windows]
• Add [EndpointSlice] support to AntreaProxy. (#1703, [@hongliangl]) [Alpha - Feature Gate: EndpointSlice]
  • EndpointSlice needs to be enabled in the K8s cluster
  • Only the "discovery.k8s.io/v1beta1" EndpointSlice API is supported
• Add support for arm/v7 and arm64 by providing Antrea Docker images for these architectures. (#1771, [@antoninbas])
  • Refer to the documentation for instructions on how to use the image
• Support IPv6 packets in Traceflow. (#1579, [@gran-vmv])
• Add the following Prometheus metrics to the the AntreaProxy implementation: "antrea_proxy_sync_proxy_rules_duration_seconds", "antrea_proxy_total_endpoints_installed", "antrea_proxy_total_endpoints_updates", "antrea_proxy_total_services_installed", "antrea_proxy_total_services_updates". (#1704, [@weiqiangt])
• Add the following Prometheus metrics to count Status updates for Antrea-native policies: "antrea_controller_acnp_status_updates", "antrea_controller_anp_status_updates". (#1801, [@antoninbas])
• Add support for TLS between the Antrea Agent FlowExporter and the FlowAggregator, using self-signed certificates. (#1649, [@zyiou])
• New Antrea Agent configuration option, "kubeAPIServerOverride", which can be used to explicitly provide an address for the K8s apiserver when the Agent is running as Pod; by default, the Agent uses the ClusterIP for the kubernetes Service. (#1735, [@anfernee])
• Provide ability to configure TLS cipher suites supported by the Antrea apiservers (Agent and Controller). (#1784, [@lzhecheng])
• Add liveness probe to Antrea Controller to ensure it is automatically restarted after a while by kubelet if it stops being responsive. (#1839, [@antoninbas])
• Document workaround to install OVS and Antrea on Windows Nodes for which the CPU does not have the required virtualization capabilities, as may be the case for cloud VMs. (#1744, [@ruicao93]) [Windows]
• Improve documentation for "noEncap" and "hybrid" traffic modes, and add information about how to use [Kube-router] to advertise Pod CIDRs to the fabric with BGP. (#1798, [@jianjuns])
• Add new NetworkPolicy testsuite based on auto-generated test cases. (#1765, [@mattfenwick])

Changed

• Change permissions for the "/var/run/antrea" directory created by the Antrea Agent on each Node to prevent non-root users from accessing it; among other things, it includes the socket file used to send CNI commands to the Agent. (#1770, [@jianjuns])
• Add multi-table support to the "antctl get ovsflows" command, to dump flows from multiple tables at once. (#1708, [@weiqiangt])
• Change the sanity check performed by the Antrea Agent to validate that the Hyper-V dependency is satisfied. (#1741, [@ruicao93])
• Periodically verify that the static iptables rules required by Antrea are present and install missing rules if any. (#1751, [@siddhant94])
• Update Mellanox/sriovnet dependency to version v1.0.2 to support OVS hardware offload to Mellanox devices with Kernel versions 5.8 and above. (#1845, [@Mmduh-483])
• Remove dependency on juju libraries, which are distributed under an LGPL v3 license. (#1796, [@antoninbas])

Fixed

• Ensure that NodePort traffic does not bypass NetworkPolicies. (#1816, [@tnqn])
  • NodePort traffic for which ExternalTrafficPolicy is set to Cluster goes through SNAT before NetworkPolicies are enforced; after SNAT the source IP is the IP of the local gateway interface (antrea-gw0)
  • Users will need to define the appropriate NetworkPolicies to allow ingress access to isolated Pods for NodePort traffic
  • This new behavior only applies to Linux Nodes using the OVS system datapath (default)
• When clearing the flow-restore-wait config for the OVS bridge after re-installing flows, ensure that the operation happened successfully and retry if anything unexpected happen; if flow-restore-wait is not cleared, the bridge will not f
  orward packets correctly. (#1730, [@tnqn])
• Stop mounting the host's kmod binary to the Antrea initContainer as it may depend on shared libraries not available in the container. (#1777, [@antoninbas])
• Fix crashes in the FlowAggregator, along with numerous spurious warnings, by updating the version of the [go-ipfix] library. (#1817, [@zyiou] [@srikartati])
• Fix issues with reference logstash configuration and improve reference Kibana dashboards for flow visualization with the FlowExporter feature. (#1727, [@zyiou])

Release v0.12.1

Changed

• More uniform mechanism in the OVS pipeline to determine whether a MAC address rewrite is needed. (#1597 #1754, @wenyingd @jianjuns)

Fixed

• Send necessary updates to Antrea Agents when a Pod's IP address is updated, as otherwise NetworkPolicies are not enforced correctly. (#1808, @Dyanngg @tnqn)
• On Antrea Agent restart, ensure that OpenFlow priorities are assigned correctly for NetworkPolicy rules, and that rules with the same tier and priority are assigned the same OpenFlow priority. (#1841, @Dyanngg)
• Do not release the OpenFlow priority assigned to a NetworkPolicy rule in case of a transient error when installing the corresponding flows, if other rules are using the same OpenFlow priority. (#1844, @Dyanngg)
• Do not delete Endpoint flows when an Endpoint is no longer used for a specific Service (or if a Service is deleted) if these flows are still required by another Service. (#1815, @weiqiangt)
• Fix AntreaProxy implementation on Windows for ClusterIP Services with endpoints outside of the cluster's Pod CIDR, by ensuring that SNAT is performed correctly. (#1824, @ruicao93) [Windows]
• More robust error handling for network adapter operations on Windows; in particular add a retry mechanism if enabling the network adapter fails. (#1736, @ruicao93) [Windows]
• When the Antrea Agent process is run using the provided PowerShell script, ensure that the Kubeconfigs used by the Agent to connect to the K8s and Antrea Controller apiservers are updated on every restart. (#1847, @ruicao93) [Windows]
• Fix bugs in IPv6 AntreaProxy implementation, notably for flow "hairpinning" and ServiceAffinity support. (#1713, @lzhecheng)
• Support non-standardized CIDRs (CIDRs for which some address bits may not have been masked off as per the prefix length) in NetworkPolicies. (#1767, @tnqn)
• Fix minimum required Linux Kernel version (4.6) in documentation. (#1757, @hongliangl)

Release v0.11.2

Fixed

• Send necessary updates to Antrea Agents when a Pod's IP address is updated, as otherwise NetworkPolicies are not enforced correctly. (#1808, @Dyanngg @tnqn)
• On Antrea Agent restart, ensure that OpenFlow priorities are assigned correctly for NetworkPolicy rules, and that rules with the same tier and priority are assigned the same OpenFlow priority. (#1841, @Dyanngg)
• Do not release the OpenFlow priority assigned to a NetworkPolicy rule in case of a transient error when installing the corresponding flows, if other rules are using the same OpenFlow priority. (#1844, @Dyanngg)
• Do not delete Endpoint flows when an Endpoint is no longer used for a specific Service (or if a Service is deleted) if these flows are still required by another Service. (#1815, @weiqiangt)
• Fix bugs in IPv6 AntreaProxy implementation, notably for flow "hairpinning" and ServiceAffinity support. (#1713, @lzhecheng)
• Support non-standardized CIDRs (CIDRs for which some address bits may not have been masked off as per the prefix length) in NetworkPolicies. (#1767, @tnqn)
• Fix minimum required Linux Kernel version (4.6) in documentation. (#1757, @hongliangl)
• Fix Agent crash when creating an Antrea-native policy with a "drop" action, while the NetworkPolicyStats feature is enabled. (#1606, @ceclinux)
• Fix Traceflow when Antrea-native policies are created with a "drop" action. (#1602, @gran-vmv @lzhecheng)
• Fix Agent crash when enabling NetworkPolicyStats and Traceflow feature together and creating an Antrea-native policy with a "drop" action. (#1615, @tnqn)
• When the destination is a Service in a Traceflow request, do not overwrite the default TCP SYN flag (needed for the packet to be processed by AntreaProxy correctly) unless the user explicitly provided a non-zero value. (#1602, @gran-vmv @lzhecheng)
• Improve handling of transient OVS errors when installing flows for policy rules in the Agent, by ensuring that retries are executed correctly. (#1667, @tnqn)