about summary refs log tree commit diff
path: root/sysdeps
Commit message (Collapse)AuthorAgeFilesLines
...
* string: Improve generic memchrAdhemerval Zanella2023-02-062-18/+5
| | | | | | | | | | | | | | | | | New algorithm read the first aligned address and mask off the unwanted bytes (this strategy is similar to arch-specific implementations used on powerpc, sparc, and sh). The loop now read word-aligned address and check using the has_eq macro. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu, and powerpc64-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* string: Improve generic strchrAdhemerval Zanella2023-02-061-5/+6
| | | | | | | | | | | | New algorithm now calls strchrnul. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu, and powerpc64-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* string: Improve generic strchrnulAdhemerval Zanella2023-02-062-6/+0
| | | | | | | | | | | | | | | | | New algorithm read the first aligned address and mask off the unwanted bytes (this strategy is similar to arch-specific implementations used on powerpc, sparc, and sh). The loop now read word-aligned address and check using the has_zero_eq function. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc64-linux-gnu, and powerpc-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* string: Improve generic strlenAdhemerval Zanella2023-02-061-4/+6
| | | | | | | | | | | | | | | | | New algorithm read the first aligned address and mask off the unwanted bytes (this strategy is similar to arch-specific implementations used on powerpc, sparc, and sh). The loop now read word-aligned address and check using the has_zero macro. Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu, and powercp64-linux-gnu by removing the arch-specific assembly implementation and disabling multi-arch (it covers both LE and BE for 64 and 32 bits). Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* Add string vectorized find and detection functionsAdhemerval Zanella2023-02-066-0/+402
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds generic string find and detection meant to be used in generic vectorized string implementation. The idea is to decompose the basic string operation so each architecture can reimplement if it provides any specialized hardware instruction. The 'string-misc.h' provides miscellaneous functions: - extractbyte: extracts the byte from an specific index. - repeat_bytes: setup an word by replicate the argument on each byte. The 'string-fza.h' provides zero byte detection functions: - find_zero_low, find_zero_all, find_eq_low, find_eq_all, find_zero_eq_low, find_zero_eq_all, and find_zero_ne_all The 'string-fzb.h' provides boolean zero byte detection functions: - has_zero: determine if any byte within a word is zero. - has_eq: determine byte equality between two words. - has_zero_eq: determine if any byte within a word is zero along with byte equality between two words. The 'string-fzi.h' provides positions for string-fza.h results: - index_first: return index of first zero byte within a word. - index_last: return index of first byte different between two words. The 'string-fzc.h' provides a combined version of fza and fzi: - index_first_zero_eq: return index of first zero byte within a word or first byte different between two words. - index_first_zero_ne: return index of first zero byte within a word or first byte equal between two words. - index_last_zero: return index of last zero byte within a word. - index_last_eq: return index of last byte different between two words. The 'string-shift.h' provides a way to mask off parts of a work based on some alignmnet (to handle unaligned arguments): - shift_find, shift_find_last. Co-authored-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
* Parameterize OP_T_THRES from memcopy.hRichard Henderson2023-02-066-14/+51
| | | | | | | | | | | | It moves OP_T_THRES out of memcopy.h to its own header and adjust each architecture that redefines it. Checked with a build and check with run-built-tests=no for all major Linux ABIs. Co-authored-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> Reviewed-by: Carlos O'Donell <carlos@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* Parameterize op_t from memcopy.hAdhemerval Zanella2023-02-063-4/+50
| | | | | | | | | | It moves the op_t definition out to an specific header, adds the attribute 'may-alias', and cleanup its duplicated definitions. Checked with a build and check with run-built-tests=no for all major Linux ABIs. Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
* AArch64: Improve SVE memcpy and memmoveWilco Dijkstra2023-02-061-20/+14
| | | | | | | | Improve SVE memcpy by copying 2 vectors if the size is small enough. This improves performance of random memcpy by ~9% on Neoverse V1, and 33-64 byte copies are ~16% faster. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* Move RETURN_TO to x86/sysdep.h and implement x86_64 version.Flavio Cruz2023-02-052-34/+17
| | | | Message-Id: <Y99nfeBrTubZL9oi@jupiter.tail36e24.ts.net>
* Remove sysdeps/mach/i386/machine-sp.hFlavio Cruz2023-02-021-30/+0
| | | | | | | This file is not used today since we end up using sysdeps/i386/htl/machine-sp.h. Getting the stack pointer does not need to be hurd specific and can go into sysdeps/<arch>. Message-Id: <Y9tpWs2WOgE/Duiq@jupiter.tail36e24.ts.net>
* hurd: Move some i386 bits to x86Samuel Thibault2023-02-0210-107/+129
| | | | As they will actually be usable on x86_64 too.
* hurd: Implement SHM_ANONSergey Bugaev2023-02-011-0/+25
| | | | | | | | | | | | | | | This adds a special SHM_ANON value that can be passed into shm_open () in place of a name. When called in this way, shm_open () will create a new anonymous shared memory file. The file will be created in the same way that other shared memory files are created (i.e., under /dev/shm/), except that it is not given a name and therefore cannot be reached from the file system, nor by other calls to shm_open (). This is accomplished by utilizing O_TMPFILE. This is intended to be compatible with FreeBSD's API of the same name. Signed-off-by: Sergey Bugaev <bugaevc@gmail.com> Message-Id: <20230130125216.6254-4-bugaevc@gmail.com>
* hurd: Implement O_TMPFILESergey Bugaev2023-02-011-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a flag that causes open () to create a new, unnamed file in the same filesystem as the given directory. The file descriptor can be simply used in the creating process as a temporary file, or shared with children processes via fork (), or sent over a Unix socket. The file can be left anonymous, in which case it will be deleted from the backing file system once all copies of the file descriptor are closed, or given a permanent name with a linkat () call, such as the following: int fd = open ("/tmp", O_TMPFILE | O_RDWR, 0700); /* Do something with the file... */ linkat (fd, "", AT_FDCWD, "/tmp/filename", AT_EMPTY_PATH); In between creating the file and linking it to the file system, it is possible to set the file content, mode, ownership, author, and other attributes, so that the file visibly appears in the file system (perhaps replacing another file) atomically, with all of its attributes already set up. The Hurd support for O_TMPFILE directly exposes the dir_mkfile RPC to user programs. Previously, dir_mkfile was used by glibc internally, in particular for implementing tmpfile (), but not exposed to user programs through a Unix-level API. O_TMPFILE was initially introduced by Linux. This implementation is intended to be compatible with the Linux implementation, except that the O_EXCL flag is not given the special meaning when used together with O_TMPFILE, unlike on Linux. Signed-off-by: Sergey Bugaev <bugaevc@gmail.com> Message-Id: <20230130125216.6254-3-bugaevc@gmail.com>
* Linux: optimize clone3 internal usageAdhemerval Zanella Netto2023-02-012-1/+32
| | | | | | | | | | | | Add an optimization to avoid calling clone3 when glibc detects that there is no kernel support. It also adds __ASSUME_CLONE3, which allows skipping this optimization and issuing the clone3 syscall directly. It does not handle the the small window between 5.3 and 5.5 for posix_spawn (CLONE_CLEAR_SIGHAND was added in 5.5). Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* aarch64: Add the clone3 wrapperAdhemerval Zanella Netto2023-02-012-0/+87
| | | | | | | | | | It follow the internal signature: extern int clone3 (struct clone_args *__cl_args, size_t __size, int (*__func) (void *__arg), void *__arg); Checked on aarch64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* linux: Add clone3 CLONE_CLEAR_SIGHAND optimization to posix_spawnAdhemerval Zanella Netto2023-02-014-26/+52
| | | | | | | | | | | | | | | | | The clone3 flag resets all signal handlers of the child not set to SIG_IGN to SIG_DFL. It allows to skip most of the sigaction calls to setup child signal handling, where previously a posix_spawn had to issue 2 times NSIG sigaction calls (one to obtain the current disposition and another to set either SIG_DFL or SIG_IGN). With POSIX_SPAWN_SETSIGDEF the child will setup the signal for the case where the disposition is SIG_IGN. The code must handle the fallback where clone3 is not available. This is done by splitting __clone_internal_fallback from __clone_internal. Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* Linux: Do not align the stack for __clone3Adhemerval Zanella Netto2023-02-012-77/+0
| | | | | | | | | | All internal callers of __clone3 should provide an already aligned stack. Removing the stack alignment in __clone3 is a net gain: it simplifies the internal function contract (mask/unmask signals) along with the arch-specific code. Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* linux: Extend internal clone3 documentationAdhemerval Zanella Netto2023-02-011-9/+1
| | | | | | | | | | | | Different than kernel, clone3 returns EINVAL for NULL struct clone_args or function pointer. This is similar to clone interface that return EINVAL for NULL function argument. It also clean up the Linux clone3.h interface, since it not currently exported. Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* linux: Do not reset signal handler in posix_spawn if it is already SIG_DFLAdhemerval Zanella Netto2023-02-011-1/+1
| | | | | | | | There is no need to issue another sigaction if the disposition is already SIG_DFL. Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* x86: Fix strncat-avx2.S reading past length [BZ #30065]Noah Goldstein2023-01-311-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Occurs when `src` has no null-term. Two cases: 1) Zero-length check is doing: ``` test %rdx, %rdx jl L(zero_len) ``` which doesn't actually check zero (was at some point `decq` and the flag never got updated). The fix is just make the flag `jle` i.e: ``` test %rdx, %rdx jle L(zero_len) ``` 2) Length check in page-cross case checking if we should continue is doing: ``` cmpq %r8, %rdx jb L(page_cross_small) ``` which means we will continue searching for null-term if length ends at the end of a page and there was no null-term in `src`. The fix is to make the flag: ``` cmpq %r8, %rdx jbe L(page_cross_small) ```
* Regenerate configure.Carlos O'Donell2023-01-312-3/+4
| | | | | | | Run using vanilla upstream autoconf 2.69. Minor whitespace change to sysdeps/loongarch/configure and sysdeps/mach/configure, and nothing else.
* sparc (64bit): Regenerate ulpsAndreas K. Hüttel2023-01-241-3/+4
| | | | | | | Linux catbus 5.15.69-gentoo #1 SMP Sat Sep 24 07:56:24 PDT 2022 sparc64 sun4v UltraSparc T5 (Niagara5) GNU/Linux gcc (Gentoo 11.3.1_p20221209 p3) 11.3.1 20221209 GNU ld (Gentoo 2.38 p4) 2.38 Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* ia64: Regenerate ulpsAndreas K. Hüttel2023-01-241-2/+2
| | | | | | | Linux guppy 5.13.0-00002-gdecb01746d6c #368 SMP Sat Aug 14 20:10:13 UTC 2021 ia64 Dual-Core Intel(R) Itanium(R) Processor 9040 GenuineIntel GNU/Linux gcc (Gentoo 12.2.1_p20221231 p8) 12.2.1 20221231 GNU ld (Gentoo 2.40 p1) 2.40 Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* x86: Cache computation for AMD architecture.Sajan Karumanchi2023-01-181-159/+45
| | | | | | | | All AMD architectures cache details will be computed based on __cpuid__ `0x8000_001D` and the reference to __cpuid__ `0x8000_0006` will be zeroed out for future architectures. Reviewed-by: Premachandra Mallappa <premachandra.mallappa@amd.com>
* AArch64: Improve strrchrWilco Dijkstra2023-01-171-25/+33
| | | | | | | | Use shrn for narrowing the mask which simplifies code and speeds up small strings. Unroll the first search loop to improve performance on large strings. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* AArch64: Optimize strnlenWilco Dijkstra2023-01-171-21/+18
| | | | | | | | Optimize strnlen using the shrn instruction and improve the main loop. Small strings are around 10% faster, large strings are 40% faster on modern CPUs. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* AArch64: Optimize strlenWilco Dijkstra2023-01-171-8/+12
| | | | | | | Optimize strlen by unrolling the main loop. Large strings are 64% faster on modern CPUs. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* AArch64: Optimize strcpyWilco Dijkstra2023-01-171-17/+19
| | | | | | Unroll the main loop. Large strings are around 20% faster on modern CPUs. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* AArch64: Improve strchrnulWilco Dijkstra2023-01-171-2/+10
| | | | | | Unroll the main loop, which improves performance slightly. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* AArch64: Optimize strchrWilco Dijkstra2023-01-171-28/+24
| | | | | | | Simplify calculation of the mask using shrn. Unroll the main loop. Small strings are 20% faster on modern CPUs. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* AArch64: Improve strlen_asimdWilco Dijkstra2023-01-171-12/+4
| | | | | | | Use shrn for the mask, merge tst+bne into cbnz, and tweak code alignment. Performance improves slightly as a result. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* AArch64: Optimize memrchrWilco Dijkstra2023-01-171-9/+11
| | | | | | Optimize the main loop - large strings are 43% faster on modern CPUs. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* AArch64: Optimize memchrWilco Dijkstra2023-01-171-13/+14
| | | | | | Optimize the main loop - large strings are 40% faster on modern CPUs. Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
* math: Suppress -O0 warnings for soft-fp fsqrt [BZ #19444]Adhemerval Zanella2023-01-111-0/+11
| | | | | | The patch suppress the same warnings from 87c266d758d29e52bfb717f90, that shows issues for microblaze, mips soft-fp, nios2, and or1k. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* Add HWCAP2_SVE_EBF16 from Linux 6.1 to AArch64 bits/hwcap.hJoseph Myers2023-01-101-0/+1
| | | | | | | Linux 6.1 adds a new AArch64 HWCAP2 value HWCAP2_SVE_EBF16; add it to the corresponding bits/hwcap.h. Tested with build-many-glibcs.py for aarch64.
* Update copyright dates with scripts/update-copyrightsJoseph Myers2023-01-067110-7110/+7110
|
* Fix ldbl-128 built-in function useJoseph Myers2023-01-053-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | Fix the following issues with built-in function use in sysdeps/ieee754/ldbl-128 and sysdeps/ieee754/float128: * fabsl used __builtin_fabsf128 unconditionally, breaking the build with GCC 6 for several architectures; it should use __builtin_fabsl with an appropriate redirection in float128_private.h. (I'm not particularly concerned with building glibc with GCC 6; rather, I want to be able to run the tgmath.h tests with GCC 6, which is a significantly different case for tgmath.h compared to GCC 7 and later because of the lack of _FloatN / _FloatNx support in the compiler, and at present running the tests with a compiler means building glibc with that compiler.) * Some (conditional) uses of built-in functions had been added to ldbl-128 without appropriate float128_private.h remapping (there was remapping for the macros controlling whether the built-in functions are used, just not for the functions themselves). * s_llrintl.c called __builtin_round not __builtin_llrintl, which is obviously wrong. Tested with build-many-glibcs.py for aarch64-linux-gnu, GCC 6 (where it fixes the glibc build) and GCC 12, and with the glibc testsuite for x86_64.
* x86: Check minimum/maximum of non_temporal_threshold [BZ #29953]H.J. Lu2023-01-031-9/+16
| | | | | | | | | The minimum non_temporal_threshold is 0x4040. non_temporal_threshold may be set to less than the minimum value when the shared cache size isn't available (e.g., in an emulator) or by the tunable. Add checks for minimum and maximum of non_temporal_threshold. This fixes BZ #29953.
* i686: Regenerate ulpsAndreas K. Hüttel2023-01-021-7/+7
| | | | Reviewed-by: Florian Weimer <fweimer@redhat.com>
* hurd getcwd: Fix memory leak on errorSamuel Thibault2023-01-021-0/+2
|
* hurd fcntl: Make LOCKED macro more robustSamuel Thibault2023-01-021-2/+3
|
* hurd: Make dl-sysdep __sbrk check __vm_allocate callSamuel Thibault2023-01-021-1/+2
| | | | | The caller won't be able to progress, but better crash than use random addr.
* htl: Drop duplicate check in __pthread_stack_allocSamuel Thibault2023-01-021-3/+0
|
* htl: Fix sem_wait race between read and gsync_waitSamuel Thibault2022-12-301-2/+2
| | | | | | | | | If the value changes between sem_wait's read and the gsync_wait call, the kernel will return KERN_INVALID_ARGUMENT, which we have to interpret as the value having already changed. This fixes applications (e.g. libgo) seeing sem_wait erroneously return KERN_INVALID_ARGUMENT.
* Linux: Pass size argument of epoll_create to the kernelFlorian Weimer2022-12-221-1/+1
| | | | | | | | | | | The kernel actually verifies it, and a garbage value in the register causes improper system call failures. Fixes commit c1c0dea38833751f36a145c32 ("Linux: Remove epoll_create, inotify_init from syscalls.list") and commit d1d23b134244d59c4d6ef2295 ("Lninux: consolidate epoll_create implementation"). Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* Define MADV_COLLAPSE from Linux 6.1Joseph Myers2022-12-222-0/+3
| | | | | | | Add the MADV_COLLAPSE constant from Linux 6.1 to bits/mman-linux.h and the hppa bits/mman.h. Tested for x86_64.
* powerpc64: Increase SIGSTKSZ and MINSIGSTKSZRajalakshmi Srinivasaraghavan2022-12-211-0/+5
| | | | | | | This patch increases the value of SIGSTKSZ and MINSIGSTKSZ for powerpc64 similar to the kernel commit 2f82ec19757f58549467db568c56e7dfff8af283 to allow further expansion of the signal stack frame size.
* Update kernel version to 6.1 in header constant testsJoseph Myers2022-12-213-4/+4
| | | | | | | | | This patch updates the kernel version in the tests tst-mman-consts.py, tst-mount-consts.py and tst-pidfd-consts.py to 6.1. (There are no new constants covered by these tests in 6.1 that need any other header changes.) Tested with build-many-glibcs.py.
* Update syscall lists for Linux 6.1Joseph Myers2022-12-201-2/+2
| | | | | Linux 6.1 has no new syscalls. Update the version number in syscall-names.list to reflect that it is still current for 6.1.
* stdio-common: Convert vfprintf and related functions to buffersFlorian Weimer2022-12-195-101/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | vfprintf is entangled with vfwprintf (of course), __printf_fp, __printf_fphex, __vstrfmon_l_internal, and the strfrom family of functions. The latter use the internal snprintf functionality, so vsnprintf is converted as well. The simples conversion is __printf_fphex, followed by __vstrfmon_l_internal and __printf_fp, and finally __vfprintf_internal and __vfwprintf_internal. __vsnprintf_internal and strfrom* are mostly consuming the new interfaces, so they are comparatively simple. __printf_fp is a public symbol, so the FILE *-based interface had to preserved. The __printf_fp rewrite does not change the actual binary-to-decimal conversion algorithm, and digits are still not emitted directly to the target buffer. However, the staging buffer now uses bytes instead of wide characters, and one buffer copy is eliminated. The changes are at least performance-neutral in my testing. Floating point printing and snprintf improved measurably, so that this Lua script for i=1,5000000 do print(i, i * math.pi) end runs about 5% faster for me. To preserve fprintf performance for a simple "%d" format, this commit has some logic changes under LABEL (unsigned_number) to avoid additional function calls. There are certainly some very easy performance improvements here: binary, octal and hexadecimal formatting can easily avoid the temporary work buffer (the number of digits can be computed ahead-of-time using one of the __builtin_clz* built-ins). Decimal formatting can use a specialized version of _itoa_word for base 10. The existing (inconsistent) width handling between strfmon and printf is preserved here. __print_fp_buffer_1 would have to use __translated_number_width to achieve ISO conformance for printf. Test expectations in libio/tst-vtables-common.c are adjusted because the internal staging buffer merges all virtual function calls into one. In general, stack buffer usage is greatly reduced, particularly for unbuffered input streams. __printf_fp can still use a large buffer in binary128 mode for %g, though. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-07-25 09:04:42 +0000