Build of pip package "cryptography" fails in QEMU 32-bit environment #30
Comments
|
So the suggested workaround is that I will add a build argument to signal that we are currently building inside a QEMU environment, and then check if the current environment is 32-bit. If both are true then we will lock the However, this will only be enabled on builds running on QEMU, native builds should not need this, since they are not affected by this problem. |
JonasAlfredsson
added a commit
that referenced
this issue
Mar 31, 2021
Our parent container is build for multiple architectures, so I think it would be nice if we followed suite. With this change we will be able to build the following platforms: - linux/amd64 - linux/386 - linux/arm64 - linux/arm/v7 - linux/mips64le - linux/s390x - linux/ppc64le However, we will only create a push function for the top 4, since the bottom 3 takes over 30 minutes to build, and none have asked for these yet. Instructions on how to set up Docker Buildx (which is required for this) can be found in [issue #28][1]. The reason for the "32-bit fix" added to the Dockerfile is a very interesting read, which can be found in [issue #30][2], but the TL;DR is that there exist a very exotic bug for some low level libraries when they try to access the filesystem while running inside a 32-bit environment that is emulated by QEMU running on a 64-bit host. For this reason we will need to pin the package version to something that is not being compiled, but this is ONLY necessary if it is inside a QEMU emulation. [1]: #28 [2]: #30
|
I will deem this one complete since the addition of the check in this commit: e941b81#diff-f7c91877446336ef59ebfaacb472e50e4e40bbe54938f232b0fd54617eb6e7dc |
JonasAlfredsson
added a commit
that referenced
this issue
Feb 18, 2022
The cryptography package fails to compile on 32-bit architectures that are QEMU emulated on a 64-bit host. We had a workaround before where we pinned the version to one that did compile without issues, but some dependency now needs a newer version which braks everything again. Next type of workaround is now to change the filesystem to a tmpfs one, which apparently does not trigger this bug. More information is available in the issue: #30
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
One really annoying thing I ran into while trying to building multiple architectures (issue #28) is that there exist a really specific problem for some low level libraries when they try to read the filesytem while running inside a 32-bit environment that is emulated by QEMU running on a 64-bit host.
There is a lot to unpack here, but it showed itself as the following errors while pip was trying to build the
cryptographypackage when running inside an emulatedlinux/arm/v7environment (e.g. Rasberry Pi) on mylinux/amd64Debian computer. This package calls oncargowhen it is being compiled, and it then fails with this very cryptic message:It looks like a network error, but in reality it appears to be related to some inode size mismatch between the 64-bit host and the 32-bit emulation. There are more threads about it that are worth a read:
There are some suggestions that you could use a
tmpfsmount forcargo's cache folder, but it looks like this will require Buildx if we are to have this during the Build step, which will make our Dockerfile less useful for those compiling with normal Docker on native hardware (since this issue only exist when building inside QEMU).The text was updated successfully, but these errors were encountered: