Inline assembly in range_decoder.h is incompatible with old Binutils #121
Comments
|
Please focus on the actual issue (failure to compile). The steps you laid out for a regular autotools project should work here. They didn't for you, so let's try figure out why:
Suggestions for improving documentation are welcome, but "it shouldn't be this hard" would apply to anyone who hits an issue. It doesn't add anything here. Anyone who files a bug ever could vent their frustration therein but it doesn't get anyone anywhere. |
I'm sorry that the experience has been negative. As you may have noticed, the home page of XZ Utils is currently quite empty but it's one of things I've been working on recently (GitHub is not the home page, it's just a place to host the repository). That happens to include writing new instructions on how to build from the Git repository. It will include more information than just running It is also true that some of the docs in the package (and thus Git repository) could be improved as some of the content is quite old and possibly a bit outdated. For example, README and INSTALL were written in a time where most would download a release tarball and build from that, and Git repository was for the few who knew how to handle it anyway. In any case I'm aware that content of the docs could be improved, it's just that there are many other things to do too. This is a hobby project, after all.
In any case, you have already correctly determined the commands to use. These are indeed quite universal steps with packages that build with Autotools. However, it seems that it didn't work. This means that it is possible that you have found a bug in XZ Utils. Thank you for reporting it! Unfortunately you didn't include the basic information:
I understand it's easy to forget those. (Previously there were (too) fancy issue templates which asked a bunch of questions. When the other maintainer disappeared, I deleted those templates as I thought it would be easy to ask in case a bug reporter doesn't mention them in their first post.) As @thesamesam wrote, the contents of
The inline assembly is used when compiling with GCC or Clang for x86-64. However, some compilers pretend to be GCC or Clang and that could be one reason for the problem. Perhaps it's possible that a compiler pretends to target x86-64 but actually doesn't and then things go wrong too. It's not useful for me guess further until you provide the hardware, OS, and compiler information. Thanks! |
|
Please re-open if more information is available. |
|
I'm now experiencing the same problem today during an attempt to bootstrap Gentoo prefix (non RAP) on an old system. It fails in stage 2 for package After a Web search, I suspect (but can't confirm) it's related one of these issues:
Strangely enough, I don't see any inline assembly in liblzma, it seems to be code generated by GCC. But even if I use GCC 4.9, the problem still occurs. A Ubuntu 16.04.7 LTS user also reported this problem to liblzma: kobolabs/liblzma#4 |
Ah, I see, the offending inline assembly is included from |
|
Please reopen the bug, and change the title to:
|
|
Thanks, nice work. I can't change the title so I'll let @Larhzu do that.
Note that we are liblzma. That's just a random repo. |
|
Right now I'm using this hack to disable inline assembly, and I can confirm it fixed the build. diff -upr xz-5.6.2/src/liblzma/rangecoder/range_decoder.h xz-5.6.2.patch/src/liblzma/rangecoder/range_decoder.h
--- xz-5.6.2/src/liblzma/rangecoder/range_decoder.h 2024-08-21 03:46:32.511055344 +0000
+++ xz-5.6.2.patch/src/liblzma/rangecoder/range_decoder.h 2024-08-21 03:47:47.072749239 +0000
@@ -52,6 +52,11 @@
# define LZMA_RANGE_DECODER_CONFIG 0
# endif
#endif
+// XXX: assembly instruction "movzw" syntax incompatible with old
+// binutils (e.g. 2.25), so disable inline assembly until we find a
+// workaround.
+// https://github.com/tukaani-project/xz/issues/121
+#define LZMA_RANGE_DECODER_CONFIG 0
// Negative RC_BIT_MODEL_TOTAL but the lowest RC_MOVE_BITS are flipped.For a complete fix we need to figure out exactly what kind of syntax is accepted by old binutils... Or at least add an autotools check that disables this feature on unsupported system. xz is a core system tool, being able to build it even on old system is crucial, as end users may want to bootstrap a newer environment. |
Larhzu
changed the title
Aha! Does it work if you search-and-replace |
|
Yes, it works with the following patch. Thanks for the quick reply. diff --git a/src/liblzma/rangecoder/range_decoder.h b/src/liblzma/rangecoder/range_decoder.h
index 31a58d1..a8aca90 100644
--- a/src/liblzma/rangecoder/range_decoder.h
+++ b/src/liblzma/rangecoder/range_decoder.h
@@ -592,13 +592,13 @@ do { \
// *_only = rc_asm_y or _n to include or exclude code marked with them
#define rc_asm_bittree(a, b, first_only, middle_only, last_only) \
first_only( \
- "movzw 2(%[probs_base]), %[prob" #a "]\n\t" \
+ "movzwl 2(%[probs_base]), %[prob" #a "]\n\t" \
"mov $2, %[symbol]\n\t" \
- "movzw 4(%[probs_base]), %[prob" #b "]\n\t" \
+ "movzwl 4(%[probs_base]), %[prob" #b "]\n\t" \
) \
middle_only( \
/* Note the scaling of 4 instead of 2: */ \
- "movzw (%[probs_base], %q[symbol], 4), %[prob" #b "]\n\t" \
+ "movzwl (%[probs_base], %q[symbol], 4), %[prob" #b "]\n\t" \
) \
last_only( \
"add %[symbol], %[symbol]\n\t" \
@@ -610,11 +610,11 @@ do { \
"cmovae %[t0], %[range]\n\t" \
\
first_only( \
- "movzw 6(%[probs_base]), %[t0]\n\t" \
+ "movzwl 6(%[probs_base]), %[t0]\n\t" \
"cmovae %[t0], %[prob" #b "]\n\t" \
) \
middle_only( \
- "movzw 2(%[probs_base], %q[symbol], 4), %[t0]\n\t" \
+ "movzwl 2(%[probs_base], %q[symbol], 4), %[t0]\n\t" \
"lea (%q[symbol], %q[symbol]), %[symbol]\n\t" \
"cmovae %[t0], %[prob" #b "]\n\t" \
) \
@@ -716,12 +716,12 @@ do { \
#define rc_asm_bittree_rev(a, b, add, dcur, dnext0, dnext1, \
first_only, middle_only, last_only) \
first_only( \
- "movzw 2(%[probs_base]), %[prob" #a "]\n\t" \
+ "movzwl 2(%[probs_base]), %[prob" #a "]\n\t" \
"xor %[symbol], %[symbol]\n\t" \
- "movzw 4(%[probs_base]), %[prob" #b "]\n\t" \
+ "movzwl 4(%[probs_base]), %[prob" #b "]\n\t" \
) \
middle_only( \
- "movzw " #dnext0 "(%[probs_base], %q[symbol], 2), " \
+ "movzwl " #dnext0 "(%[probs_base], %q[symbol], 2), " \
"%[prob" #b "]\n\t" \
) \
\
@@ -731,11 +731,11 @@ do { \
"cmovae %[t0], %[range]\n\t" \
\
first_only( \
- "movzw 6(%[probs_base]), %[t0]\n\t" \
+ "movzwl 6(%[probs_base]), %[t0]\n\t" \
"cmovae %[t0], %[prob" #b "]\n\t" \
) \
middle_only( \
- "movzw " #dnext1 "(%[probs_base], %q[symbol], 2), %[t0]\n\t" \
+ "movzwl " #dnext1 "(%[probs_base], %q[symbol], 2), %[t0]\n\t" \
"cmovae %[t0], %[prob" #b "]\n\t" \
) \
\
@@ -788,7 +788,7 @@ do { \
uint32_t t_index; \
\
__asm__( \
- "movzw (%[probs_base], %q[symbol], 2), %[prob]\n\t" \
+ "movzwl (%[probs_base], %q[symbol], 2), %[prob]\n\t" \
"mov %[symbol], %[index]\n\t" \
\
"add %[dest], %[t2]\n\t" \
@@ -844,7 +844,7 @@ do { \
"and %[offset], %[match_bit]\n\t" \
"add %[match_bit], %[symbol]\n\t" \
\
- "movzw (%[probs_base], %q[symbol], 2), %[prob]\n\t" \
+ "movzwl (%[probs_base], %q[symbol], 2), %[prob]\n\t" \
\
"add %[symbol], %[symbol]\n\t" \
\ |
biergaizi
added a commit
to biergaizi/xz
that referenced
this issue
Aug 22, 2024
…oject#121. Support for instruction "movzw" without suffix in "GNU as" was added in commit [1] and stabilized in binutils 2.27, released in August 2016. Earlier systems don't accept this instruction without a suffix, making range_decoder.h's inline assembly unable to build on old systems such as Ubuntu 16.04, creating error messages like: lzma_decoder.c: Assembler messages: lzma_decoder.c:371: Error: no such instruction: `movzw 2(%r11),%esi' lzma_decoder.c:373: Error: no such instruction: `movzw 4(%r11),%edi' lzma_decoder.c:388: Error: no such instruction: `movzw 6(%r11),%edx' lzma_decoder.c:398: Error: no such instruction: `movzw (%r11,%r14,4),%esi' This has been reported by at least 3 different users. Since xz and liblzma are base system tools and are often required in system upgrades themselves, compatibility with old systems shall be kept. This commit changes "movzw" to "movzwl" for compatibility. [1] https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c07315e0c610e0e3317b4c02266f81793df253d2 Suggested-By: Lasse Collin <lasse.collin@tukaani.org> Tested-By: Yifeng Li <tomli@tomli.me> Signed-off-by: Yifeng Li <tomli@tomli.me>
|
I just opened Pull Request #136 with the patch attached. |
biergaizi
added a commit
to biergaizi/xz
that referenced
this issue
Aug 22, 2024
…oject#121. Support for instruction "movzw" without suffix in "GNU as" was added in commit [1] and stabilized in binutils 2.27, released in August 2016. Earlier systems don't accept this instruction without a suffix, making range_decoder.h's inline assembly unable to build on old systems such as Ubuntu 16.04, creating error messages like: lzma_decoder.c: Assembler messages: lzma_decoder.c:371: Error: no such instruction: `movzw 2(%r11),%esi' lzma_decoder.c:373: Error: no such instruction: `movzw 4(%r11),%edi' lzma_decoder.c:388: Error: no such instruction: `movzw 6(%r11),%edx' lzma_decoder.c:398: Error: no such instruction: `movzw (%r11,%r14,4),%esi' This has been reported by at least 3 different users. Since xz and liblzma are base system tools and are often required in system upgrades themselves, compatibility with old systems shall be kept. This commit changes "movzw" to "movzwl" for compatibility. [1] https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c07315e0c610e0e3317b4c02266f81793df253d2 Suggested-by: Lasse Collin <lasse.collin@tukaani.org> Tested-by: Yifeng Li <tomli@tomli.me> Signed-off-by: Yifeng Li <tomli@tomli.me>
Yes. I occasionally test on Slackware 10.1 (from 2005) which has GCC 3.3.2. (Easy with Autotools but I haven't gotten CMake or Meson+Ninja to run.) That's 32-bit x86 though. I hadn't realized that there are this kind of code compatibility issues at x86-64 side nowadays too. Ubuntu 16 is almost fresh still. ;-) Somehow binutils compatibility didn't cross my mind when this issue was created (perhaps something in the first message of this thread distracted me).
I edited the commit message slightly and merged it. I verified that your diff to the code was identical to what I had prepared locally yesterday. Thanks! |
10 tasks
Larhzu
pushed a commit
that referenced
this issue
Sep 6, 2024
Support for instruction "movzw" without suffix in "GNU as" was
added in commit [1] and stabilized in binutils 2.27, released
in August 2016. Earlier systems don't accept this instruction
without a suffix, making range_decoder.h's inline assembly
unable to build on old systems such as Ubuntu 16.04, creating
error messages like:
lzma_decoder.c: Assembler messages:
lzma_decoder.c:371: Error: no such instruction: `movzw 2(%r11),%esi'
lzma_decoder.c:373: Error: no such instruction: `movzw 4(%r11),%edi'
lzma_decoder.c:388: Error: no such instruction: `movzw 6(%r11),%edx'
lzma_decoder.c:398: Error: no such instruction: `movzw (%r11,%r14,4),%esi'
Change "movzw" to "movzwl" for compatibility.
[1] https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c07315e0c610e0e3317b4c02266f81793df253d2
Suggested-by: Lasse Collin <lasse.collin@tukaani.org>
Tested-by: Yifeng Li <tomli@tomli.me>
Signed-off-by: Yifeng Li <tomli@tomli.me>
Fixes: 3182a33
Fixes: #121
Closes: #136
(cherry picked from commit 6cd7c86)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I just wanted to compile the latest src for xz
in the absence of clear instructions on the opening page of the git repo, and the INSTALL file being a TLDR nightmare, after years of experience of compiling stuff, i used my intuition to try to compile it
so i
git clone https://github.com/tukaani-project/xz --depth=1(--depth=1, because i don't want to download all the commit history, which should not be necessary to build)
then i ran ./autogen.sh
and then ran ./configure
then ran make
(pretty standard universal compile steps, that work with most src using cmake/make)
then i got this error
lzma/lzma_decoder.c: Assembler messages:
lzma/lzma_decoder.c:371: Error: no such instruction:
movzw 2(%r15),%esi' lzma/lzma_decoder.c:373: Error: no such instruction:movzw 4(%r15),%edi'lzma/lzma_decoder.c:388: Error: no such instruction:
movzw 6(%r15),%edx' lzma/lzma_decoder.c:398: Error: no such instruction:movzw (%r15,%r14,4),%esi'lzma/lzma_decoder.c:413: Error: no such instruction:
movzw 2(%r15,%r14,4),%edx' lzma/lzma_decoder.c:424: Error: no such instruction:movzw (%r15,%r14,4),%edi'lzma/lzma_decoder.c:439: Error: no such instruction:
movzw 2(%r15,%r14,4),%edx' lzma/lzma_decoder.c:450: Error: no such instruction:movzw (%r15,%r14,4),%esi'lzma/lzma_decoder.c:465: Error: no such instruction:
movzw 2(%r15,%r14,4),%edx' lzma/lzma_decoder.c:476: Error: no such instruction:movzw (%r15,%r14,4),%edi'lzma/lzma_decoder.c:491: Error: no such instruction:
movzw 2(%r15,%r14,4),%edx' lzma/lzma_decoder.c:502: Error: no such instruction:movzw (%r15,%r14,4),%esi'lzma/lzma_decoder.c:517: Error: no such instruction:
movzw 2(%r15,%r14,4),%edx' lzma/lzma_decoder.c:528: Error: no such instruction:movzw (%r15,%r14,4),%edi'lzma/lzma_decoder.c:543: Error: no such instruction:
movzw 2(%r15,%r14,4),%edx' lzma/lzma_decoder.c:421: Error: no such instruction:movzw 2(%rax),%edi'lzma/lzma_decoder.c:423: Error: no such instruction:
movzw 4(%rax),%r10d' lzma/lzma_decoder.c:438: Error: no such instruction:movzw 6(%rax),%ecx'lzma/lzma_decoder.c:448: Error: no such instruction:
movzw (%rax,%rdx,4),%edi' lzma/lzma_decoder.c:463: Error: no such instruction:movzw 2(%rax,%rdx,4),%ecx'lzma/lzma_decoder.c:430: Error: no such instruction:
movzw 2(%r15),%esi' lzma/lzma_decoder.c:432: Error: no such instruction:movzw 4(%r15),%edi'lzma/lzma_decoder.c:447: Error: no such instruction:
movzw 6(%r15),%eax' lzma/lzma_decoder.c:457: Error: no such instruction:movzw (%r15,%rdx,4),%esi'lzma/lzma_decoder.c:472: Error: no such instruction:
movzw 2(%r15,%rdx,4),%eax' lzma/lzma_decoder.c:483: Error: no such instruction:movzw (%r15,%rdx,4),%edi'lzma/lzma_decoder.c:498: Error: no such instruction:
movzw 2(%r15,%rdx,4),%eax' lzma/lzma_decoder.c:509: Error: no such instruction:movzw (%r15,%rdx,4),%esi'lzma/lzma_decoder.c:524: Error: no such instruction:
movzw 2(%r15,%rdx,4),%eax' lzma/lzma_decoder.c:535: Error: no such instruction:movzw (%r15,%rdx,4),%edi'lzma/lzma_decoder.c:550: Error: no such instruction:
movzw 2(%r15,%rdx,4),%eax' lzma/lzma_decoder.c:487: Error: no such instruction:movzw (%r15,%rdi,2),%r13d'lzma/lzma_decoder.c:379: Error: no such instruction:
movzw (%r15,%r14,2),%r10d' lzma/lzma_decoder.c:411: Error: no such instruction:movzw (%r15,%r14,2),%r10d'lzma/lzma_decoder.c:443: Error: no such instruction:
movzw (%r15,%r14,2),%r10d' lzma/lzma_decoder.c:475: Error: no such instruction:movzw (%r15,%r14,2),%r10d'lzma/lzma_decoder.c:507: Error: no such instruction:
movzw (%r15,%r14,2),%r10d' lzma/lzma_decoder.c:539: Error: no such instruction:movzw (%r15,%r14,2),%r10d'lzma/lzma_decoder.c:571: Error: no such instruction:
movzw (%r15,%r14,2),%r10d' lzma/lzma_decoder.c:603: Error: no such instruction:movzw (%r15,%r14,2),%r10d'lzma/lzma_decoder.c:421: Error: no such instruction:
movzw 2(%rax),%edi' lzma/lzma_decoder.c:423: Error: no such instruction:movzw 4(%rax),%r10d'lzma/lzma_decoder.c:438: Error: no such instruction:
movzw 6(%rax),%ecx' lzma/lzma_decoder.c:448: Error: no such instruction:movzw (%rax,%rdx,4),%edi'lzma/lzma_decoder.c:463: Error: no such instruction:
movzw 2(%rax,%rdx,4),%ecx' lzma/lzma_decoder.c:627: Error: no such instruction:movzw 2(%rax),%esi'lzma/lzma_decoder.c:629: Error: no such instruction:
movzw 4(%rax),%edi' lzma/lzma_decoder.c:644: Error: no such instruction:movzw 6(%rax),%edx'lzma/lzma_decoder.c:654: Error: no such instruction:
movzw (%rax,%r14,4),%esi' lzma/lzma_decoder.c:669: Error: no such instruction:movzw 2(%rax,%r14,4),%edx'lzma/lzma_decoder.c:627: Error: no such instruction:
movzw 2(%rax),%esi' lzma/lzma_decoder.c:629: Error: no such instruction:movzw 4(%rax),%edi'lzma/lzma_decoder.c:644: Error: no such instruction:
movzw 6(%rax),%edx' lzma/lzma_decoder.c:654: Error: no such instruction:movzw (%rax,%r14,4),%esi'lzma/lzma_decoder.c:669: Error: no such instruction:
movzw 2(%rax,%r14,4),%edx' lzma/lzma_decoder.c:506: Error: no such instruction:movzw 2(%r10),%esi'lzma/lzma_decoder.c:508: Error: no such instruction:
movzw 4(%r10),%edi' lzma/lzma_decoder.c:523: Error: no such instruction:movzw 6(%r10),%edx'lzma/lzma_decoder.c:533: Error: no such instruction:
movzw 8(%r10,%r14,2),%esi' lzma/lzma_decoder.c:548: Error: no such instruction:movzw 12(%r10,%r14,2),%edx'lzma/lzma_decoder.c:559: Error: no such instruction:
movzw 16(%r10,%r14,2),%edi' lzma/lzma_decoder.c:574: Error: no such instruction:movzw 24(%r10,%r14,2),%edx'lzma/lzma_decoder.c:421: Error: no such instruction:
movzw 2(%rcx),%edi' lzma/lzma_decoder.c:423: Error: no such instruction:movzw 4(%rcx),%r10d'lzma/lzma_decoder.c:438: Error: no such instruction:
movzw 6(%rcx),%edx' lzma/lzma_decoder.c:448: Error: no such instruction:movzw (%rcx,%rax,4),%edi'lzma/lzma_decoder.c:463: Error: no such instruction:
movzw 2(%rcx,%rax,4),%edx' lzma/lzma_decoder.c:474: Error: no such instruction:movzw (%rcx,%rax,4),%r10d'lzma/lzma_decoder.c:489: Error: no such instruction:
movzw 2(%rcx,%rax,4),%edx' lzma/lzma_decoder.c:500: Error: no such instruction:movzw (%rcx,%rax,4),%edi'lzma/lzma_decoder.c:515: Error: no such instruction:
movzw 2(%rcx,%rax,4),%edx' lzma/lzma_decoder.c:526: Error: no such instruction:movzw (%rcx,%rax,4),%r10d'lzma/lzma_decoder.c:541: Error: no such instruction:
movzw 2(%rcx,%rax,4),%edx' lzma/lzma_decoder.c:552: Error: no such instruction:movzw (%rcx,%rax,4),%edi'lzma/lzma_decoder.c:567: Error: no such instruction:
movzw 2(%rcx,%rax,4),%edx' lzma/lzma_decoder.c:578: Error: no such instruction:movzw (%rcx,%rax,4),%r10d'lzma/lzma_decoder.c:593: Error: no such instruction:
movzw 2(%rcx,%rax,4),%edx' lzma/lzma_decoder.c:627: Error: no such instruction:movzw 2(%rdi),%ecx'lzma/lzma_decoder.c:629: Error: no such instruction:
movzw 4(%rdi),%esi' lzma/lzma_decoder.c:644: Error: no such instruction:movzw 6(%rdi),%eax'lzma/lzma_decoder.c:654: Error: no such instruction:
movzw (%rdi,%r14,4),%ecx' lzma/lzma_decoder.c:669: Error: no such instruction:movzw 2(%rdi,%r14,4),%eax'lzma/lzma_decoder.c:680: Error: no such instruction:
movzw (%rdi,%r14,4),%esi' lzma/lzma_decoder.c:695: Error: no such instruction:movzw 2(%rdi,%r14,4),%eax'lzma/lzma_decoder.c:706: Error: no such instruction:
movzw (%rdi,%r14,4),%ecx' lzma/lzma_decoder.c:721: Error: no such instruction:movzw 2(%rdi,%r14,4),%eax'lzma/lzma_decoder.c:732: Error: no such instruction:
movzw (%rdi,%r14,4),%esi' lzma/lzma_decoder.c:747: Error: no such instruction:movzw 2(%rdi,%r14,4),%eax'lzma/lzma_decoder.c:758: Error: no such instruction:
movzw (%rdi,%r14,4),%ecx' lzma/lzma_decoder.c:773: Error: no such instruction:movzw 2(%rdi,%r14,4),%eax'lzma/lzma_decoder.c:784: Error: no such instruction:
movzw (%rdi,%r14,4),%esi' lzma/lzma_decoder.c:799: Error: no such instruction:movzw 2(%rdi,%r14,4),%eax'Makefile:1482: recipe for target 'liblzma_la-lzma_decoder.lo' failed
what am i doing wrong and why are there no compile instructions on the main page?
i suggest you stick some basic compile instructions instead of that wall of text on the main page.
The text was updated successfully, but these errors were encountered: