Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Windows] Finish removing Docker and userspace kube-proxy support #6255

Merged
merged 4 commits into from
Apr 24, 2024
Merged

[Windows] Finish removing Docker and userspace kube-proxy support #6255

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6c82414
[Windows] Finish removing Docker and userspace kube-proxy support
antoninbas Apr 18, 2024
411cda4
Rename Antrea Windows YAML manifests
antoninbas Apr 23, 2024
fe6e1ac
Address review comments
antoninbas Apr 23, 2024
9b7cc64
Update WINDOWS_YAML_SUFFIX in ci/jenkins/test.sh
antoninbas Apr 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -417,8 +417,8 @@ build-migrator:
manifest:
@echo "===> Generating dev manifest for Antrea <==="
$(CURDIR)/hack/generate-standard-manifests.sh --mode dev --out build/yamls
$(CURDIR)/hack/generate-manifest-windows.sh --mode dev > build/yamls/antrea-windows-containerd.yml
$(CURDIR)/hack/generate-manifest-windows.sh --mode dev --include-ovs > build/yamls/antrea-windows-containerd-with-ovs.yml
$(CURDIR)/hack/generate-manifest-windows.sh --mode dev > build/yamls/antrea-windows.yml
$(CURDIR)/hack/generate-manifest-windows.sh --mode dev --include-ovs > build/yamls/antrea-windows-with-ovs.yml
$(CURDIR)/hack/update-checksum-windows.sh
$(CURDIR)/hack/generate-manifest-flow-aggregator.sh --mode dev > build/yamls/flow-aggregator.yml

Expand Down
0 ...ls/antrea-windows-containerd-with-ovs.yml → build/yamls/antrea-windows-with-ovs.yml
View file
Open in desktop
File renamed without changes.
0 build/yamls/antrea-windows-containerd.yml → build/yamls/antrea-windows.yml
View file
Open in desktop
File renamed without changes.
6 changes: 3 additions & 3 deletions ci/jenkins/test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -375,7 +375,7 @@ function revert_snapshot_windows {
function build_and_deliver_antrea_windows_and_linux_containerd_images {
echo "====== Cleanup Antrea Installation Before Delivering Antrea Windows and Antrea Linux containerd Images ====="
clean_antrea
kubectl delete -f ${WORKDIR}/antrea-windows-containerd-with-ovs.yml --ignore-not-found=true || true
kubectl delete -f ${WORKDIR}/antrea-windows-with-ovs.yml --ignore-not-found=true || true
XinShuYang marked this conversation as resolved.
Show resolved Hide resolved
kubectl delete -f ${WORKDIR}/kube-proxy-windows-containerd.yml --ignore-not-found=true || true
kubectl delete daemonset antrea-agent -n kube-system --ignore-not-found=true || true
kubectl delete -f ${WORKDIR}/antrea.yml --ignore-not-found=true || true
Expand All @@ -388,11 +388,11 @@ function build_and_deliver_antrea_windows_and_linux_containerd_images {
${PRINT_DOCKER_STATUS}
export_govc_env_var
# Enable verbose log for troubleshooting.
sed -i "s/--v=0/--v=4/g" build/yamls/antrea.yml build/yamls/antrea-windows-containerd-with-ovs.yml
sed -i "s/--v=0/--v=4/g" build/yamls/antrea.yml build/yamls/antrea-windows-with-ovs.yml

echo "====== Updating yaml files to enable proxyAll ======"
KUBE_API_SERVER=$(kubectl --kubeconfig=$KubeConfigFile config view -o jsonpath='{.clusters[0].cluster.server}')
sed -i "s|.*kubeAPIServerOverride: \"\"| kubeAPIServerOverride: \"${KUBE_API_SERVER}\"|g" build/yamls/antrea.yml build/yamls/antrea-windows-containerd-with-ovs.yml
sed -i "s|.*kubeAPIServerOverride: \"\"| kubeAPIServerOverride: \"${KUBE_API_SERVER}\"|g" build/yamls/antrea.yml build/yamls/antrea-windows-with-ovs.yml

cp -f build/yamls/*.yml $WORKDIR
set +e
Expand Down
10 changes: 5 additions & 5 deletions docs/design/windows-design.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,11 +190,11 @@ these scenarios:

## Antrea and OVS Management on Windows

While we provide different installation methods for Windows, the recommended one starting with
Antrea v1.13 is to use the `antrea-windows-containerd-with-ovs.yml` manifest. With this method, the
antrea-agent process and the OVS daemons (ovsdb-server and ovs-vswitchd) run as a Pod on Windows
worker Nodes, and are managed by a DaemonSet. This installation method relies on
[Windows HostProcess Pod](https://kubernetes.io/docs/tasks/configure-pod-container/create-hostprocess-pod/)
While we provide different installation methods for Windows, the recommended one
is to use the `antrea-windows-with-ovs.yml` manifest. With this method, the
antrea-agent process and the OVS daemons (ovsdb-server and ovs-vswitchd) run as
a Pod on Windows worker Nodes, and are managed by a DaemonSet. This installation
method relies on [Windows HostProcess Pod](https://kubernetes.io/docs/tasks/configure-pod-container/create-hostprocess-pod/)
support.

## Traffic walkthrough
Expand Down
331 changes: 53 additions & 278 deletions docs/windows.md
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions hack/release/prepare-assets.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,8 +110,8 @@ export CONTROLLER_IMG_NAME=antrea/antrea-controller-ubuntu
./hack/generate-standard-manifests.sh --mode release --out "$OUTPUT_DIR"

export IMG_NAME=antrea/antrea-windows
./hack/generate-manifest-windows.sh --mode release > "$OUTPUT_DIR"/antrea-windows-containerd.yml
./hack/generate-manifest-windows.sh --mode release --include-ovs > "$OUTPUT_DIR"/antrea-windows-containerd-with-ovs.yml
./hack/generate-manifest-windows.sh --mode release > "$OUTPUT_DIR"/antrea-windows.yml
./hack/generate-manifest-windows.sh --mode release --include-ovs > "$OUTPUT_DIR"/antrea-windows-with-ovs.yml

export IMG_NAME=antrea/flow-aggregator
./hack/generate-manifest-flow-aggregator.sh --mode release > "$OUTPUT_DIR"/flow-aggregator.yml
Expand Down
4 changes: 2 additions & 2 deletions hack/update-checksum-windows.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,5 +36,5 @@ for file in ${MANIFESTS[@]}; do
sed -i.bak "s/windows-config-checksum-placeholder/${checksum_windows_config}/g" ${file}
done

sed -i.bak "s/agent-windows-checksum-placeholder/${checksum_containerd}/g" ${YAMLS_DIR}/antrea-windows-containerd.yml
sed -i.bak "s/agent-windows-checksum-placeholder/${checksum_containerd_with_ovs}/g" ${YAMLS_DIR}/antrea-windows-containerd-with-ovs.yml
sed -i.bak "s/agent-windows-checksum-placeholder/${checksum_containerd}/g" ${YAMLS_DIR}/antrea-windows.yml
sed -i.bak "s/agent-windows-checksum-placeholder/${checksum_containerd_with_ovs}/g" ${YAMLS_DIR}/antrea-windows-with-ovs.yml
9 changes: 6 additions & 3 deletions hack/windows/Prepare-AntreaAgent.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,13 @@ This script prepares environment needed by antrea-agent which includes:
provide the proxy for Kubernetes Services.

.PARAMETER InstallKubeProxy
Specifies whether kube-proxy interface is included in the installation. If false, this interface will not
be installed on the host.
[DEPRECATED] Specifies whether kube-proxy interface is included in the installation.

.PARAMETER RunOVSServices
Specifies whether the OVS userspace daemons should be started as Windows services.
#>
Param(
[parameter(Mandatory = $false)] [bool] $InstallKubeProxy = $true,
[parameter(Mandatory = $false)] [bool] $InstallKubeProxy = $false,
[parameter(Mandatory = $false)] [bool] $RunOVSServices= $true
)

Expand Down Expand Up @@ -48,6 +50,7 @@ if ($RunOVSServices -eq $true) {
}
# Prepare service network interface for kube-proxy.
if ($InstallKubeProxy -eq $true) {
Write-Host "Running Antrea with kube-proxy is no longer supported, this parameter will be removed soon"
Write-Host "Preparing service network interface for kube-proxy..."
& $PrepareServiceInterfaceScript
}
45 changes: 15 additions & 30 deletions hack/windows/Prepare-Node.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ script from the sig-windows-tools repo release page: https://github.com/kubernet
Kubernetes version to download and use

.PARAMETER InstallKubeProxy
Install kube-proxy
[DEPRECATED] Install kube-proxy

.PARAMETER InstallOVS
Install OVS
Expand All @@ -26,11 +26,12 @@ Install OVS
The node ip used by kubelet

.PARAMETER ContainerRuntime
Container runtime that Kubernetes will use. (docker or containerd)
[DEPRECATED] Container runtime that Kubernetes will use (docker or containerd).
Starting with Antrea v2.0, only containerd is supported.

.PARAMETER InstallOVSUserspace
Specifies whether OVS userspace processes are included in the installation. If false, these processes will not
be installed as a Windows service on the host.
Specifies whether OVS userspace processes are included in the installation. If false, these
processes will not be installed as a Windows service on the host.

.EXAMPLE
PS> .\Prepare-Node.ps1 -KubernetesVersion v1.27.0 -NodeIP 192.168.1.10 -ContainerRuntime containerd
Expand All @@ -43,8 +44,8 @@ Param(
[parameter(Mandatory = $false)] [switch] $InstallKubeProxy = $false,
[parameter(Mandatory = $false)] [switch] $InstallOVS = $false,
[parameter(Mandatory = $false, HelpMessage="Kubernetes download")] [string] $KubernetesURL="dl.k8s.io",
[parameter(HelpMessage="Container runtime that Kubernets will use")] [ValidateSet("containerd", "docker")] [string] $ContainerRuntime = "containerd",
[parameter(Mandatory = $false)] [bool] $InstallOVSUserspace = $true
[parameter(Mandatory = $false)] [ValidateSet("containerd", "docker")] [string] $ContainerRuntime = "containerd",
[parameter(Mandatory = $false)] [bool] $InstallOVSUserspace = $true
)
$ErrorActionPreference = 'Stop'

Expand All @@ -64,15 +65,13 @@ If (Get-Service kubelet -ErrorAction SilentlyContinue) {
}

if ($ContainerRuntime -eq "docker") {
if (-not(Test-Path "//./pipe/docker_engine")) {
Write-Error "Docker service was not detected - please install and start Docker before calling Prepare-Node.ps1 with -ContainerRuntime docker"
exit 1
}
} elseif ($ContainerRuntime -eq "containerd") {
if (-not(Test-Path "//./pipe/containerd-containerd")) {
Write-Error "Containerd service was not detected - please install and start Containerd before calling Prepare-Node.ps1 with -ContainerRuntime containerd"
exit 1
}
Write-Error "Docker container runtime is no longer supported"
exit 1
}

if (-not(Test-Path "//./pipe/containerd-containerd")) {
Write-Error "Containerd service was not detected - please install and start containerd before calling Prepare-Node.ps1"
exit 1
}

if (!$KubernetesVersion.StartsWith("v")) {
Expand All @@ -94,13 +93,6 @@ $env:Path += ";$global:KubernetesPath"
DownloadFile $kubeletBinPath "https:/$KubernetesURL/$KubernetesVersion/bin/windows/amd64/kubelet.exe"
DownloadFile "$global:KubernetesPath\kubeadm.exe" "https:/$KubernetesURL/$KubernetesVersion/bin/windows/amd64/kubeadm.exe"

if ($ContainerRuntime -eq "docker") {
Write-Host "Registering wins service"
DownloadFile "$global:KubernetesPath\wins.exe" https://github.com/rancher/wins/releases/download/v0.0.4/wins.exe
wins.exe srv app run --register
start-service rancher-wins
}


mkdir -force C:\var\log\kubelet
mkdir -force C:\var\lib\kubelet\etc\kubernetes
Expand All @@ -111,15 +103,8 @@ New-Item -path C:\var\lib\kubelet\etc\kubernetes\pki -type SymbolicLink -value C
$StartKubeletFileContent = '$FileContent = Get-Content -Path "/var/lib/kubelet/kubeadm-flags.env"
$global:KubeletArgs = $FileContent.Trim("KUBELET_KUBEADM_ARGS=`"")'+ [Environment]::NewLine

if ($ContainerRuntime -eq "docker") {
$StartKubeletFileContent +=[Environment]::NewLine +'$netId = docker network ls -f name=host --format "{{ .ID }}"

if ($netId.Length -lt 1) {
docker network create -d nat host
}' + [Environment]::NewLine
}

if ($InstallKubeProxy) {
Write-Host "Running Antrea with kube-proxy is no longer supported, this parameter will be removed soon"
$StartKubeletFileContent += [Environment]::NewLine + '& C:\k\Prepare-ServiceInterface.ps1 -InterfaceAlias "HNS Internal NIC"' + [Environment]::NewLine
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this line? I think we can just print the error message for InstallKubeProxy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to wait for a couple more releases before removing kube-proxy support from the scripts completely (see PR description). Do you think we should just remove them now?
I personally think that would be acceptable, let's see if anyone else has an opinion on that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wenyingd @luolanzone let me know what you think
We can either keep the current version, or drop KubeProxy support completely from the scripts

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both works for me, if it's removed from K8s 1.26, I guess it should be safe to remove as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep it as is since there is no real drawback and we are close to the release date. I will remove it post v2 and we can have the change in v2.1. Sounds acceptable?

}

Expand Down
1 change: 1 addition & 0 deletions hack/windows/Start-AntreaAgent.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ if ($LastExitCode) {
}

if ($StartKubeProxy) {
Write-Host "Running Antrea with kube-proxy is no longer supported, this parameter will be removed soon"
luolanzone marked this conversation as resolved.
Show resolved Hide resolved
Write-Host "Starting kube-proxy..."
if (!(Start-KubeProxy -KubeProxy $KubernetesHome\kube-proxy.exe -KubeConfig $KubeConfig)) {
Write-Host "Failed to start kube-proxy, exit"
Expand Down
Loading