about summary refs log tree commit diff
Commit message (Collapse)AuthorAgeFilesLines
* S390: Influence hwcaps/stfle via GLIBC_TUNABLES.Stefan Liebler2023-02-0712-64/+423
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch enables the option to influence hwcaps and stfle bits used by the s390 specific ifunc-resolvers. The currently x86-specific tunable glibc.cpu.hwcaps is also used on s390x to achieve the task. In addition the user can also set a CPU arch-level like z13 instead of single HWCAP and STFLE features. Note that the tunable only handles the features which are really used in the IFUNC-resolvers. All others are ignored as the values are only used inside glibc. Thus we can influence: - HWCAP_S390_VXRS (z13) - HWCAP_S390_VXRS_EXT (z14) - HWCAP_S390_VXRS_EXT2 (z15) - STFLE_MIE3 (z15) The influenced hwcap/stfle-bits are stored in the s390-specific cpu_features struct which also contains reserved fields for future usage. The ifunc-resolvers and users of stfle bits are adjusted to use the information from cpu_features struct. On 31bit, the ELF_MACHINE_IRELATIVE macro is now also defined. Otherwise the new ifunc-resolvers segfaults as they depend on the not yet processed_rtld_global_ro@GLIBC_PRIVATE relocation.
* string: Hook up the default implementation on test-strrchrAdhemerval Zanella2023-02-061-14/+24
| | | | | | And remove SIMPLE_STRRCHR, which is not used anywhere. Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* string: Hook up the default implementation on test-memrchrAdhemerval Zanella2023-02-061-0/+7
| | | | Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* string: Hook up the default implementation on test-memchrAdhemerval Zanella2023-02-061-11/+20
| | | | | | And remove SIMPLE_MEMCHR, which is not used anywhere. Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* string: Hook up the default implementation on test-strcpyAdhemerval Zanella2023-02-061-10/+24
| | | | Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* string: Hook up the default implementation on test-stpcpyAdhemerval Zanella2023-02-061-11/+21
| | | | Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* string: Hook up the default implementation on test-strncmpAdhemerval Zanella2023-02-061-0/+16
| | | | Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* string: Hook up the default implementation on test-strcmpAdhemerval Zanella2023-02-061-0/+22
| | | | Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* string: Hook up the default implementation on test-strchrAdhemerval Zanella2023-02-061-15/+38
| | | | | | Also remove the simple_STRCHR, which can be easily replaced. Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* string: Hook up the default implementation on test-strnlenAdhemerval Zanella2023-02-061-11/+24
| | | | | | Also remove the SIMPLE_STRNLEN, which is not used anywhere. Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* string: Hook up the default implementation on test-strlenAdhemerval Zanella2023-02-061-17/+14
| | | | | | | Also remove the simple_STRLEN and builtin_strlen, which are not used anywhere. Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* riscv: Add string-fza.h and string-fzi.hAdhemerval Zanella2023-02-062-0/+146
| | | | | | | | | | It uses the bitmanip extension to optimize index_fist and index_last with clz/ctz (using generic implementation that routes to compiler builtin) and orc.b to check null bytes. Checked the string test on riscv64 user mode. Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* sh: Add string-fzb.hAdhemerval Zanella2023-02-061-0/+55
| | | | | | Use the SH cmp/str on has_{zero,eq,zero_eq}. Checked on sh4-linux-gnu.
* powerpc: Add string-fza.hRichard Henderson2023-02-061-0/+71
| | | | | | | | | | | While ppc has the more important string functions in assembly, there are still a few generic routines used. Use the Power 6 CMPB insn for testing of zeros. Checked on powerpc64le-linux-gnu. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* arm: Add string-fza.hRichard Henderson2023-02-061-0/+67
| | | | | | | | | | | While arm has the more important string functions in assembly, there are still a few generic routines used. Use the UQSUB8 insn for testing of zeros. Checked on armv7-linux-gnueabihf Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* alpha: Add string-fza, string-fzb.h, string-fzi.h, and string-shift.hRichard Henderson2023-02-064-0/+218
| | | | | | | | | | | | | While alpha has the more important string functions in assembly, there are still a few for find the generic routines are used. Use the CMPBGE insn, via the builtin, for testing of zeros. Use a simplified expansion of __builtin_ctz when the insn isn't available. Checked on alpha-linux-gnu. Co-authored-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* hppa: Add string-fza.h, string-fzc.h, and string-fzi.hRichard Henderson2023-02-063-0/+250
| | | | | | | | | | | | | Use UXOR,SBZ to test for a zero byte within a word. While we can get semi-decent code out of asm-goto, we would do slightly better with a compiler builtin. For index_zero et al, sequential testing of bytes is less expensive than any tricks that involve a count-leading-zeros insn that we don't have. Checked on hppa-linux-gnu. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* hppa: Add memcopy.hRichard Henderson2023-02-061-0/+42
| | | | | | | | | | | | | | | GCC's combine pass cannot merge (x >> c | y << (32 - c)) into a double-word shift unless (1) the subtract is in the same basic block and (2) the result of the subtract is used exactly once. Neither condition is true for any use of MERGE. By forcing the use of a double-word shift, we not only reduce contention on SAR, but also allow the setting of SAR to be hoisted outside of a loop. Checked on hppa-linux-gnu. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* string: Improve generic strrchr with memrchr and strlenAdhemerval Zanella2023-02-061-17/+1
| | | | | | | | | | | Now that both strlen and memrchr have word vectorized implementation, it should be faster to implement strrchr based on memrchr over the string length instead of calling strchr on a loop. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu, and powerpc64-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits).
* string: Improve generic memrchrAdhemerval Zanella2023-02-061-157/+39
| | | | | | | | | | | | | | New algorithm read the lastaligned address and mask off the unwanted bytes. The loop now read word-aligned address and check using the has_eq macro. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu, and powerpc64-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* string: Improve generic strnlen with memchrAdhemerval Zanella2023-02-064-152/+27
| | | | | | | | It also cleanups the multiple inclusion by leaving the ifunc implementation to undef the weak_alias and libc_hidden_def. Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* string: Improve generic memchrAdhemerval Zanella2023-02-063-143/+56
| | | | | | | | | | | | | | | | | New algorithm read the first aligned address and mask off the unwanted bytes (this strategy is similar to arch-specific implementations used on powerpc, sparc, and sh). The loop now read word-aligned address and check using the has_eq macro. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu, and powerpc64-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* string: Improve generic strcpyAdhemerval Zanella2023-02-061-1/+5
| | | | | | | | | | | | | Now that stpcpy is vectorized based on op_t, it should be better to call it instead of strlen plus memcpy. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64-linux-gnu, and powerpc-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* string: Improve generic stpcpyAdhemerval Zanella2023-02-061-6/+86
| | | | | | | | | | | | | | | | | | | | | | | It follows the strategy: - Align the destination on word boundary using byte operations. - If source is also word aligned, read a word per time, check for null (using has_zero from string-fzb.h), and write the remaining bytes. - If source is not word aligned, loop by aligning the source, and merging the result of two reads. Similar to aligned case, check for null with has_zero, and write the remaining bytes if null is found. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64-linux-gnu, and powerpc-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* string: Improve generic strncmpAdhemerval Zanella2023-02-061-37/+101
| | | | | | | | | | | | | | | | | | | | | It follows the strategy: - Align the first input to word boundary using byte operations. - If second input is also word aligned, read a word per time, check for null (using has_zero), and check final words using byte operation. - If second input is not word aligned, loop by aligning the source, and merge the result of two reads. Similar to aligned case, check for null with has_zero, and check final words using byte operation. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64-linux-gnu, and powerpc-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* string: Improve generic strcmpAdhemerval Zanella2023-02-061-16/+94
| | | | | | | | | | | | | | | | | | | | | It follows the strategy: - Align the first input to word boundary using byte operations. - If second input is also word aligned, read a word per time, check for null (using has_zero), and check final words using byte operation. - If second input is not word aligned, loop by aligning the source, and merging the result of two reads. Similar to aligned case, check for null with has_zero, and check final words using byte operation. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64-linux-gnu, and powerpc-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* string: Improve generic strchrAdhemerval Zanella2023-02-062-161/+14
| | | | | | | | | | | | New algorithm now calls strchrnul. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu, and powerpc64-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* string: Improve generic strchrnulAdhemerval Zanella2023-02-063-139/+22
| | | | | | | | | | | | | | | | | New algorithm read the first aligned address and mask off the unwanted bytes (this strategy is similar to arch-specific implementations used on powerpc, sparc, and sh). The loop now read word-aligned address and check using the has_zero_eq function. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64-linux-gnu, and powerpc-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* string: Improve generic strlenAdhemerval Zanella2023-02-062-74/+28
| | | | | | | | | | | | | | | | | New algorithm read the first aligned address and mask off the unwanted bytes (this strategy is similar to arch-specific implementations used on powerpc, sparc, and sh). The loop now read word-aligned address and check using the has_zero macro. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu, and powercp64-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* Add string vectorized find and detection functionsAdhemerval Zanella2023-02-066-0/+402
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds generic string find and detection meant to be used in generic vectorized string implementation. The idea is to decompose the basic string operation so each architecture can reimplement if it provides any specialized hardware instruction. The 'string-misc.h' provides miscellaneous functions: - extractbyte: extracts the byte from an specific index. - repeat_bytes: setup an word by replicate the argument on each byte. The 'string-fza.h' provides zero byte detection functions: - find_zero_low, find_zero_all, find_eq_low, find_eq_all, find_zero_eq_low, find_zero_eq_all, and find_zero_ne_all The 'string-fzb.h' provides boolean zero byte detection functions: - has_zero: determine if any byte within a word is zero. - has_eq: determine byte equality between two words. - has_zero_eq: determine if any byte within a word is zero along with byte equality between two words. The 'string-fzi.h' provides positions for string-fza.h results: - index_first: return index of first zero byte within a word. - index_last: return index of first byte different between two words. The 'string-fzc.h' provides a combined version of fza and fzi: - index_first_zero_eq: return index of first zero byte within a word or first byte different between two words. - index_first_zero_ne: return index of first zero byte within a word or first byte equal between two words. - index_last_zero: return index of last zero byte within a word. - index_last_eq: return index of last byte different between two words. The 'string-shift.h' provides a way to mask off parts of a work based on some alignmnet (to handle unaligned arguments): - shift_find, shift_find_last. Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* Parameterize OP_T_THRES from memcopy.hRichard Henderson2023-02-067-17/+51
| | | | | | | | | | | | It moves OP_T_THRES out of memcopy.h to its own header and adjust each architecture that redefines it. Checked with a build and check with run-built-tests=no for all major Linux ABIs. Co-authored-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> Reviewed-by: Carlos O'Donell <carlos@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* Parameterize op_t from memcopy.hAdhemerval Zanella2023-02-064-5/+50
| | | | | | | | | | It moves the op_t definition out to an specific header, adds the attribute 'may-alias', and cleanup its duplicated definitions. Checked with a build and check with run-built-tests=no for all major Linux ABIs. Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* Replace rawmemchr (s, '\0') with strchrWilco Dijkstra2023-02-0624-48/+34
| | | | | | | | | Almost all uses of rawmemchr find the end of a string. Since most targets use a generic implementation, replacing it with strchr is better since that is optimized by compilers into strlen (s) + s. Also fix the generic rawmemchr implementation to use a cast to unsigned char in the if statement. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* AArch64: Improve SVE memcpy and memmoveWilco Dijkstra2023-02-061-20/+14
| | | | | | | | Improve SVE memcpy by copying 2 vectors if the size is small enough. This improves performance of random memcpy by ~9% on Neoverse V1, and 33-64 byte copies are ~16% faster. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* Account for grouping in printf width (bug 30068)Carlos O'Donell2023-02-063-5/+73
| | | | | | | | | This is a partial fix for mishandling of grouping when formatting integers. It properly computes the width in the presence of grouping characters when the width is larger than the number of significant digits. The precision related issue is documented in bug 23432. Co-authored-by: Andreas Schwab <schwab@suse.de>
* Move RETURN_TO to x86/sysdep.h and implement x86_64 version.Flavio Cruz2023-02-052-34/+17
| | | | Message-Id: <Y99nfeBrTubZL9oi@jupiter.tail36e24.ts.net>
* Remove pthread-pi-defines.symAndreas Schwab2023-02-032-11/+1
| | | | | It became unused with the removal of the assembler implementation of the pthread functions.
* stdlib: tests: don't double-define _FORTIFY_SOURCESam James2023-02-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | If using -D_FORITFY_SOURCE=3 (in my case, I've patched GCC to add =3 instead of =2 (we've done =2 for years in Gentoo)), building glibc tests will fail on testmb like: ``` <command-line>: error: "_FORTIFY_SOURCE" redefined [-Werror] <built-in>: note: this is the location of the previous definition cc1: all warnings being treated as errors make[2]: *** [../o-iterator.mk:9: /var/tmp/portage/sys-libs/glibc-2.36/work/build-x86-x86_64-pc-linux-gnu-nptl/stdlib/testmb.o] Error 1 make[2]: *** Waiting for unfinished jobs.... ``` It's just because we're always setting -D_FORTIFY_SOURCE=2 rather than unsetting it first. If F_S is already 2, it's harmless, but if it's another value (say, 1, or 3), the compiler will bawk. (I'm not aware of a reason this couldn't be tested with =3, but the toolchain support is limited for that (too new), and we want to run the tests everywhere possible.) Signed-off-by: Sam James <sam@gentoo.org> Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
* LoongArch: Add new relocation types.caiyinyu2023-02-031-0/+40
|
* Remove sysdeps/mach/i386/machine-sp.hFlavio Cruz2023-02-021-30/+0
| | | | | | | This file is not used today since we end up using sysdeps/i386/htl/machine-sp.h. Getting the stack pointer does not need to be hurd specific and can go into sysdeps/<arch>. Message-Id: <Y9tpWs2WOgE/Duiq@jupiter.tail36e24.ts.net>
* cdefs: Limit definition of fortification macrosSiddhesh Poyarekar2023-02-021-2/+4
| | | | | | | | | | | | Define the __glibc_fortify and other macros only when __FORTIFY_LEVEL > 0. This has the effect of not defining these macros on older C90 compilers that do not have support for variable length argument lists. Also trim off the trailing backslashes from the definition of __glibc_fortify and __glibc_fortify_n macros. Signed-off-by: Siddhesh Poyarekar <siddhesh@sourceware.org> Reviewed-by: Florian Weimer <fweimer@redhat.com>
* hurd: Move some i386 bits to x86Samuel Thibault2023-02-0210-107/+129
| | | | As they will actually be usable on x86_64 too.
* Remove support setting custom demuxers during signal handling.Flavio Cruz2023-02-011-17/+0
| | | | | We seem to call only into the exception and message server routines. Message-Id: <Y9dpRZs3QYk2oZm+@jupiter.tail36e24.ts.net>
* hurd: Implement SHM_ANONSergey Bugaev2023-02-013-4/+51
| | | | | | | | | | | | | | | This adds a special SHM_ANON value that can be passed into shm_open () in place of a name. When called in this way, shm_open () will create a new anonymous shared memory file. The file will be created in the same way that other shared memory files are created (i.e., under /dev/shm/), except that it is not given a name and therefore cannot be reached from the file system, nor by other calls to shm_open (). This is accomplished by utilizing O_TMPFILE. This is intended to be compatible with FreeBSD's API of the same name. Signed-off-by: Sergey Bugaev <bugaevc@gmail.com> Message-Id: <20230130125216.6254-4-bugaevc@gmail.com>
* hurd: Implement O_TMPFILESergey Bugaev2023-02-012-0/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a flag that causes open () to create a new, unnamed file in the same filesystem as the given directory. The file descriptor can be simply used in the creating process as a temporary file, or shared with children processes via fork (), or sent over a Unix socket. The file can be left anonymous, in which case it will be deleted from the backing file system once all copies of the file descriptor are closed, or given a permanent name with a linkat () call, such as the following: int fd = open ("/tmp", O_TMPFILE | O_RDWR, 0700); /* Do something with the file... */ linkat (fd, "", AT_FDCWD, "/tmp/filename", AT_EMPTY_PATH); In between creating the file and linking it to the file system, it is possible to set the file content, mode, ownership, author, and other attributes, so that the file visibly appears in the file system (perhaps replacing another file) atomically, with all of its attributes already set up. The Hurd support for O_TMPFILE directly exposes the dir_mkfile RPC to user programs. Previously, dir_mkfile was used by glibc internally, in particular for implementing tmpfile (), but not exposed to user programs through a Unix-level API. O_TMPFILE was initially introduced by Linux. This implementation is intended to be compatible with the Linux implementation, except that the O_EXCL flag is not given the special meaning when used together with O_TMPFILE, unlike on Linux. Signed-off-by: Sergey Bugaev <bugaevc@gmail.com> Message-Id: <20230130125216.6254-3-bugaevc@gmail.com>
* hurd: Consolidate file_name_lookup implementationSergey Bugaev2023-02-012-24/+37
| | | | | | | | | | | Instead of __file_name_lookup_at delegating to __file_name_lookup in simple cases, make __file_name_lookup_at deal with both cases, and have __file_name_lookup simply wrap __file_name_lookup_at. This factorizes handling the empy name case. Signed-off-by: Sergey Bugaev <bugaevc@gmail.com> Message-Id: <20230130125216.6254-2-bugaevc@gmail.com>
* Linux: optimize clone3 internal usageAdhemerval Zanella Netto2023-02-013-1/+37
| | | | | | | | | | | | Add an optimization to avoid calling clone3 when glibc detects that there is no kernel support. It also adds __ASSUME_CLONE3, which allows skipping this optimization and issuing the clone3 syscall directly. It does not handle the the small window between 5.3 and 5.5 for posix_spawn (CLONE_CLEAR_SIGHAND was added in 5.5). Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* aarch64: Add the clone3 wrapperAdhemerval Zanella Netto2023-02-012-0/+87
| | | | | | | | | | It follow the internal signature: extern int clone3 (struct clone_args *__cl_args, size_t __size, int (*__func) (void *__arg), void *__arg); Checked on aarch64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* linux: Add clone3 CLONE_CLEAR_SIGHAND optimization to posix_spawnAdhemerval Zanella Netto2023-02-017-27/+238
| | | | | | | | | | | | | | | | | The clone3 flag resets all signal handlers of the child not set to SIG_IGN to SIG_DFL. It allows to skip most of the sigaction calls to setup child signal handling, where previously a posix_spawn had to issue 2 times NSIG sigaction calls (one to obtain the current disposition and another to set either SIG_DFL or SIG_IGN). With POSIX_SPAWN_SETSIGDEF the child will setup the signal for the case where the disposition is SIG_IGN. The code must handle the fallback where clone3 is not available. This is done by splitting __clone_internal_fallback from __clone_internal. Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* Linux: Do not align the stack for __clone3Adhemerval Zanella Netto2023-02-013-77/+3
| | | | | | | | | | All internal callers of __clone3 should provide an already aligned stack. Removing the stack alignment in __clone3 is a net gain: it simplifies the internal function contract (mask/unmask signals) along with the arch-specific code. Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>