All tasks fail on logging to a Windows drive (Kubernetes-Multipass-Microk8s on Windows host) #17129
Labels
affected_version:2.1
Issues Reported for 2.1
area:logging
good first issue
kind:bug
This is a clearly a bug
Stale Bug Report
Apache Airflow version: 2.1.0
Kubernetes version (if you are using kubernetes) (use
kubectl version
): v1.19.7Environment:
uname -a
):On the Windows host we have installed Microk8s. The installer installs MultiPass with an Ubuntu VM. Airflow is installed using the helm chart from https://airflow.apache.org . We use the KubernetesExecutor and tasks are started as pods in-cluster.
Specific to this environment is that we instructed airflow to write the log files to a folder inside the Ubuntu VM, which itself is mapped to a folder on the windows host using
multipass mount
.What happened:
Every task that gets started in a Kubernetes pod fails immediately with the following exception:
What you expected to happen:
When I run the exact same setup on a Mac OS X host it works without problem, so I expect the tasks to run in a pod and logs to be written.
The problem is very likely the following:
The proposed solution then is to use log folder naming conventions that are supported on all filesystems including on Windows hosts.
Anything else we need to know:
This is an example of issue #10388 . While solving this issue will not make airflow compatible with Windows, at least it will allow us to use a Microk8s VM on our development machines and test environment regardless of whether the developer is using a Mac, a Windows PC or Ubuntu.
The text was updated successfully, but these errors were encountered: