about summary refs log tree commit diff
path: root/sysdeps
Commit message (Collapse)AuthorAgeFilesLines
* x86: Remove low-level lock optimizationFlorian Weimer2021-04-213-89/+0
| | | | | | | | | | | | | | | The current approach is to do this optimizations at a higher level, in generic code, so that single-threaded cases can be specifically targeted. Furthermore, using IS_IN (libc) as a compile-time indicator that all locks are private is no longer correct once process-shared lock implementations are moved into libc. The generic <lowlevellock.h> is not compatible with assembler code (obviously), so it's necessary to remove two long-unused #includes. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move internal symbol __mutex_aconf into libcFlorian Weimer2021-04-213-0/+102
| | | | | | | | This is in preparation of moving the mutex code into libc. __pthread_tunables_init is now called via __pthread_early_init. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* pthread: Introduce __pthread_early_initFlorian Weimer2021-04-211-0/+29
| | | | | | | This function is called from __libc_early_init to initialize the pthread subsystem. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move rwlock functions with forwarders into libcFlorian Weimer2021-04-2163-192/+294
| | | | | | | | | | | | | | | The forwarders were only used internally, so new symbol versions are needed. All symbols are moved at once because the forwarders are no-ops if libpthread is not loaded, leading to inconsistencies in case of a partial migration. The symbols __pthread_rwlock_rdlock, __pthread_rwlock_unlock, __pthread_rwlock_wrlock, pthread_rwlock_rdlock, pthread_rwlock_unlock, pthread_rwlock_wrlock have been moved using scripts/move-symbol-to-libc.py. The __ symbol variants are turned into compat symbols, which is why they do not receive a GLIBC_2.34 version.
* nptl: Move pthread_key_delete into libc.Florian Weimer2021-04-2161-29/+64
| | | | | | | The symbol was moved using scripts/move-symbol-to-libc.py. tss_delete (still in libpthread) uses the __pthread_key_create alias, so that is now exported under GLIBC_PRIVATE.
* nptl: Move pthread_setspecific, __pthread_setspecific into libcFlorian Weimer2021-04-2163-61/+128
| | | | The symbols have been moved using scripts/move-symbol-to-libc.py.
* nptl: Move pthread_getspecific, __pthread_getspecific into libcFlorian Weimer2021-04-2163-61/+128
| | | | The symbols have been moved using scripts/move-symbol-to-libc.py.
* nptl: Move pthread_key_create, __pthread_key_create into libcFlorian Weimer2021-04-2163-61/+128
| | | | The symbols have been moved using scripts/move-symbol-to-libc.py.
* nptl: Move part of TCB initialization from libpthread to __tls_init_tpFlorian Weimer2021-04-212-29/+37
| | | | | | | | | | | | | | | | | | | This initalization should only happen once for the main thread's TCB. At present, auditors can achieve this by not linking against libpthread. If libpthread becomes part of libc, doing this initialization in libc would happen for every audit namespace, or too late (if it happens from the main libc only). That's why moving this code into ld.so seems the right thing to do, right after the TCB initialization. For !__ASSUME_SET_ROBUST_LIST ports, this also moves the symbol __set_robust_list_avail into ld.so, as __nptl_set_robust_list_avail. It also turned into a proper boolean flag. Inline the __pthread_initialize_pids function because it seems no longer useful as a separate function. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* elf: Introduce __tls_init_tp for second-phase TCB initializationFlorian Weimer2021-04-212-0/+35
| | | | | | | | | | TLS_INIT_TP is processor-specific, so it is not a good place to put thread library initialization code (it would have to be repeated for all CPUs). Introduce __tls_init_tp as a separate function, to be called immediately after TLS_INIT_TP. Move the existing stack list setup code for NPTL to this function. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* Remove pthread_key_create-related internals from libc-lock.hFlorian Weimer2021-04-213-48/+7
| | | | | And libc-lockP.h. This is no longer used because all internal TLS use goes directly to the thread descriptor/TCB or uses ELF TLS.
* dlfcn: dlerror needs to call free from the base namespace [BZ #24773]Florian Weimer2021-04-211-0/+7
| | | | | | | | | | | | Calling free directly may end up freeing a pointer allocated by the dynamic loader using malloc from libc.so in the base namespace using the allocator from libc.so in a secondary namespace, which results in crashes. This commit redirects the free call through GLRO and the dynamic linker, to reach the correct namespace. It also cleans up the dlerror handling along the way, so that pthread_setspecific is no longer needed (which avoids triggering bug 24774).
* dlfcn: Failures after dlmopen should not terminate process [BZ #24772]Florian Weimer2021-04-211-0/+9
| | | | | | | | | | | | | | | | | | | Commit 9e78f6f6e7134a5f299cc8de77370218f8019237 ("Implement _dl_catch_error, _dl_signal_error in libc.so [BZ #16628]") has the side effect that distinct namespaces, as created by dlmopen, now have separate implementations of the rtld exception mechanism. This means that the call to _dl_catch_error from libdl in a secondary namespace does not actually install an exception handler because the thread-local variable catch_hook in the libc.so copy in the secondary namespace is distinct from that of the base namepace. As a result, a dlsym/dlopen/... failure in a secondary namespace terminates the process with a dynamic linker error because it looks to the exception handler mechanism as if no handler has been installed. This commit restores GLRO (dl_catch_error) and uses it to set the handler in the base namespace. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Invoke the set_robust_list system call directly in forkFlorian Weimer2021-04-212-10/+2
| | | | | | This removes one of the pthread forwarder functions. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move pthread_setcanceltype into libcFlorian Weimer2021-04-2130-30/+0
| | | | | | | | No new symbol version is required because there was a forwarder. The symbol has been moved using scripts/move-symbol-to-libc.py. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move pthread_setcancelstate into libcFlorian Weimer2021-04-2134-42/+6
| | | | | | | | No new symbol version is required because there was a forwarder. The symbol has been moved using scripts/move-symbol-to-libc.py. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move pthread_exit into libcFlorian Weimer2021-04-2131-31/+1
| | | | | | | | | | | | | | The pthread_exit symbol was moved using scripts/move-symbol-to-libc.py. No new symbol version is needed because there was a forwarder. The new tests nptl/tst-pthread_exit-nothreads and nptl/tst-pthread_exit-nothreads-static exercise the scenario that pthread_exit is called without libpthread having been linked in. This is not possible for the generic code, so these tests do not live in sysdeps/pthread for now. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move __nptl_deallocate_tsd into libcFlorian Weimer2021-04-212-6/+0
| | | | | | | This prepares moving pthread_exit, and later the pthread_key_create infrastructure. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move internal __nptl_nthreads variable into libcFlorian Weimer2021-04-212-13/+2
| | | | Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* csu: Move calling main out of __libc_start_main_implFlorian Weimer2021-04-212-0/+112
| | | | | | | This code depends on whether glibc has unwinding support for a particular port. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move __pthread_unwind_next into libcFlorian Weimer2021-04-2171-162/+86
| | | | | | | | | | | | | | | | | | | It's necessary to stub out __libc_disable_asynccancel and __libc_enable_asynccancel via rtld-stubbed-symbols because the new direct references to the unwinder result in symbol conflicts when the rtld exception handling from libc is linked in during the construction of librtld.map. unwind-forcedunwind.c is merged into unwind-resume.c. libc now needs the functions that were previously only used in libpthread. The GLIBC_PRIVATE exports of __libc_longjmp and __libc_siglongjmp are no longer needed, so switch them to hidden symbols. The symbol __pthread_unwind_next has been moved using scripts/move-symbol-to-libc.py. Reviewed-by: Adhemerva Zanella <adhemerval.zanella@linaro.org>
* nptl: Move pthread_once and __pthread_once into libcFlorian Weimer2021-04-2164-72/+131
| | | | | | | | | | | | | And also the fork generation counter, __fork_generation. This eliminates the need for __fork_generation_pointer. call_once remains in libpthread and calls the exported __pthread_once symbol. pthread_once and __pthread_once have been moved using scripts/move-symbol-to-libc.py. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move __pthread_cleanup_upto into libcFlorian Weimer2021-04-213-12/+4
| | | | | | | | This internal symbol is used as part of the longjmp implementation. Rename the file from nptl/pt-cleanup.c to nptl/pthread_cleanup_upto.c so that the pt-* files remain restricted to libpthread. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* x86: Restore compile-time check for shadow stack pointer in longjmpAdhemerval Zanella2021-04-211-0/+39
|
* nptl: Remove longjmp, siglongjmp from libpthreadFlorian Weimer2021-04-2130-224/+28
| | | | | | | | | | | | | | The definitions in libc are sufficient, the forwarders are no longer needed. The symbols have been moved using scripts/move-symbol-to-libc.py. s390-linux-gnu and s390x-linux-gnu need a new version placeholder to keep the GLIBC_2.19 symbol version in libpthread. Tested on i386-linux-gnu, powerpc64le-linux-gnu, s390x-linux-gnu, x86_64-linux-gnu. Built with build-many-glibcs.py. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move legacy cancelation handling into libc as compat symbolsFlorian Weimer2021-04-2161-116/+128
| | | | | | | | | | | | | | | This affects _pthread_cleanup_pop, _pthread_cleanup_pop_restore, _pthread_cleanup_push, _pthread_cleanup_push_defer. The symbols have been moved using scripts/move-symbol-to-libc.py. No new symbol versions are added because the symbols are turned into compatibility symbols at the same time. __pthread_cleanup_pop and __pthread_cleanup_push are added as GLIBC_PRIVATE symbols because they are also used internally, for glibc's own cancellation handling. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move legacy unwinding implementation into libcFlorian Weimer2021-04-213-58/+31
| | | | | | | | | | | | It is still used internally. Since unwinding is now available unconditionally, avoid indirect calls through function pointers loaded from the stack by inlining the non-cancellation cleanup code. This avoids a regression in security hardening. The out-of-line __libc_cleanup_routine implementation is no longer needed because the inline definition is now static __always_inline. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move __pthread_cleanup_routine into libcFlorian Weimer2021-04-2161-29/+64
| | | | Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Move pthread_mutex_consistent into libcFlorian Weimer2021-04-2165-65/+109
| | | | | | And deprecated pthread_mutex_consistent_np, its old name. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* elf: Remove lazy tlsdesc relocation related codeSzabolcs Nagy2021-04-214-4/+0
| | | | | | | | | | | Remove generic tlsdesc code related to lazy tlsdesc processing since lazy tlsdesc relocation is no longer supported. This includes removing GL(dl_load_lock) from _dl_make_tlsdesc_dynamic which is only called at load time when that lock is already held. Added a documentation comment too. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* x86: Optimize strlen-avx2.SNoah Goldstein2021-04-192-214/+334
| | | | | | | | | | No bug. This commit optimizes strlen-avx2.S. The optimizations are mostly small things but they add up to roughly 10-30% performance improvement for strlen. The results for strnlen are bit more ambiguous. test-strlen, test-strnlen, test-wcslen, and test-wcsnlen are all passing. Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
* x86: Optimize strlen-evex.SNoah Goldstein2021-04-191-264/+317
| | | | | | | | | | No bug. This commit optimizes strlen-evex.S. The optimizations are mostly small things but they add up to roughly 10-30% performance improvement for strlen. The results for strnlen are bit more ambiguous. test-strlen, test-strnlen, test-wcslen, and test-wcsnlen are all passing. Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
* x86: Optimize less_vec evex and avx512 memset-vec-unaligned-erms.SNoah Goldstein2021-04-195-27/+74
| | | | | | | | No bug. This commit adds optimized cased for less_vec memset case that uses the avx512vl/avx512bw mask store avoiding the excessive branches. test-memset and test-wmemset are passing. Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
* x86-64: Require BMI2 for strchr-avx2.SH.J. Lu2021-04-192-5/+11
| | | | | | | | | | | | | | | | | Since strchr-avx2.S updated by commit 1f745ecc2109890886b161d4791e1406fdfc29b8 Author: noah <goldstein.w.n@gmail.com> Date: Wed Feb 3 00:38:59 2021 -0500 x86-64: Refactor and improve performance of strchr-avx2.S uses sarx: c4 e2 72 f7 c0 sarx %ecx,%eax,%eax for strchr-avx2 family functions, require BMI2 in ifunc-impl-list.c and ifunc-avx2.h.
* x86-64: Require BMI2 for __strlen_evex and __strnlen_evexH.J. Lu2021-04-191-2/+4
| | | | | | | | | | | | | | | | | Since __strlen_evex and __strnlen_evex added by commit 1fd8c163a83d96ace1ff78fa6bac7aee084f6f77 Author: H.J. Lu <hjl.tools@gmail.com> Date: Fri Mar 5 06:24:52 2021 -0800 x86-64: Add ifunc-avx2.h functions with 256-bit EVEX use sarx: c4 e2 6a f7 c0 sarx %edx,%eax,%eax require BMI2 for __strlen_evex and __strnlen_evex in ifunc-impl-list.c. ifunc-avx2.h already requires BMI2 for EVEX implementation.
* x86: Update large memcpy case in memmove-vec-unaligned-erms.Snoah2021-04-161-73/+265
| | | | | | | | | | | | | | No Bug. This commit updates the large memcpy case (no overlap). The update is to perform memcpy on either 2 or 4 contiguous pages at once. This 1) helps to alleviate the affects of false memory aliasing when destination and source have a close 4k alignment and 2) In most cases and for most DRAM units is a modestly more efficient access pattern. These changes are a clear performance improvement for VEC_SIZE =16/32, though more ambiguous for VEC_SIZE=64. test-memcpy, test-memccpy, test-mempcpy, test-memmove, and tst-memmove-overflow all pass. Signed-off-by: Noah Goldstein <goldstein.w.n@gmail.com>
* powerpc: Add missing registers to clobbers list for syscalls [BZ #27623]Matheus Castanho2021-04-161-3/+6
| | | | | | | | | | | | | | | | | | | | | | | | Some registers that can be clobbered by the kernel during a syscall are not listed on the clobbers list in sysdeps/unix/sysv/linux/powerpc/sysdep.h. For syscalls using sc: - XER is zeroed by the kernel on exit For syscalls using scv: - XER is zeroed by the kernel on exit - Different from the sc case, most CR fields can be clobbered (according to the ELF ABI and the Linux kernel's syscall ABI for powerpc (linux/Documentation/powerpc/syscall64-abi.rst) The same should apply to vsyscalls, which effectively execute a function call but are not currently adding these registers as clobbers either. These are likely not causing issues today, but they should be added to the clobbers list just in case things change on the kernel side in the future. Reported-by: Nicholas Piggin <npiggin@gmail.com> Reviewed-by: Nicholas Piggin <npiggin@gmail.com> Reviewed-by: Raphael M Zinsly <rzinsly@linux.ibm.com>
* misc: syslog: Assume MSG_NOSIGNAL support (BZ #17144)Adhemerval Zanella2021-04-151-4/+0
| | | | | | | | MSG_NOSIGNAL was added on POSIX 2008 and Hurd seems to support it. The SIGPIPE handling also makes the implementation not thread-safe (due the sigaction usage). Checked on x86_64-linux-gnu.
* io: Move file timestamps tests out of LinuxAdhemerval Zanella2021-04-155-229/+1
| | | | | | | | | | | | Now that libsupport abstract Linux possible missing support (either due FS limitation that can't handle 64 bit timestamp or architectures that do not handle values larger than unsigned 32 bit values) the tests can be turned generic. Checked on x86_64-linux-gnu and i686-linux-gnu. I also built the tests for i686-gnu. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
* s390: Update ulpsStefan Liebler2021-04-151-3/+3
| | | | | Required after 9acda61d94acc "Fix the inaccuracy of j0f/j1f/y0f/y1f [BZ #14469, #14470, #14471, #14472]".
* i386: Remove lazy tlsdesc relocation related codeSzabolcs Nagy2021-04-153-391/+2
| | | | | | | | | | Like in commit e75711ebfa976d5468ec292282566a18b07e4d67 for x86_64, remove unused lazy tlsdesc relocation processing code: _dl_tlsdesc_resolve_abs_plus_addend _dl_tlsdesc_resolve_rel _dl_tlsdesc_resolve_rela _dl_tlsdesc_resolve_hold
* x86_64: Remove lazy tlsdesc relocation related codeSzabolcs Nagy2021-04-154-219/+2
| | | | | _dl_tlsdesc_resolve_rela and _dl_tlsdesc_resolve_hold are only used for lazy tlsdesc relocation processing which is no longer supported.
* i386: Avoid lazy relocation of tlsdesc [BZ #27137]Szabolcs Nagy2021-04-151-42/+34
| | | | | | | | | | | Lazy tlsdesc relocation is racy because the static tls optimization and tlsdesc management operations are done without holding the dlopen lock. This similar to the commit b7cf203b5c17dd6d9878537d41e0c7cc3d270a67 for aarch64, but it fixes a different race: bug 27137. On i386 the code is a bit more complicated than on x86_64 because both rel and rela relocs are supported.
* x86_64: Avoid lazy relocation of tlsdesc [BZ #27137]Szabolcs Nagy2021-04-151-5/+14
| | | | | | | | | | | | | Lazy tlsdesc relocation is racy because the static tls optimization and tlsdesc management operations are done without holding the dlopen lock. This similar to the commit b7cf203b5c17dd6d9878537d41e0c7cc3d270a67 for aarch64, but it fixes a different race: bug 27137. Another issue is that ld auditing ignores DT_BIND_NOW and thus tries to relocate tlsdesc lazily, but that does not work in a BIND_NOW module due to missing DT_TLSDESC_PLT. Unconditionally relocating tlsdesc at load time fixes this bug 27721 too.
* ARC: Update ulpsVineet Gupta2021-04-142-25/+29
| | | | | | Needed after 43576de04afc6 Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
* Remove PR_TAGGED_ADDR_ENABLE from sys/prctl.hSzabolcs Nagy2021-04-141-4/+0
| | | | | | | | | | | | | | The value of PR_TAGGED_ADDR_ENABLE was incorrect in the installed headers and the prctl command macros were missing that are needed for it to be useful (PR_SET_TAGGED_ADDR_CTRL). Linux headers have the definitions since 5.4 so it's widely available, we don't need to repeat these definitions. The remaining definitions are from Linux 5.10. To build glibc with --enable-memory-tagging, Linux 5.4 headers and binutils 2.33.1 or newer is needed. Reviewed-by: DJ Delorie <dj@redhat.com>
* linux: sysconf: Use a more explicit maximum_ARG_MAXAdhemerval Zanella2021-04-131-1/+1
|
* linux: sysconf: limit _SC_MAX_ARG to 6 MiB (BZ #25305)Michal Nazarewicz2021-04-131-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since Linux 4.13, kernel limits the maximum command line arguments length to 6 MiB [1]. Normally the limit is still quarter of the maximum stack size but if that limit exceeds 6 MiB it's clamped down. glibc's __sysconf implementation for Linux platform is not aware of this limitation and for stack sizes of over 24 MiB it returns higher ARG_MAX than Linux will actually accept. This can be verified by executing the following application on Linux 4.13 or newer: #include <stdio.h> #include <string.h> #include <sys/resource.h> #include <sys/time.h> #include <unistd.h> int main(void) { const struct rlimit rlim = { 40 * 1024 * 1024, 40 * 1024 * 1024 }; if (setrlimit(RLIMIT_STACK, &rlim) < 0) { perror("setrlimit: RLIMIT_STACK"); return 1; } printf("ARG_MAX : %8ld\n", sysconf(_SC_ARG_MAX)); printf("63 * 100 KiB: %8ld\n", 63L * 100 * 1024); printf("6 MiB : %8ld\n", 6L * 1024 * 1024); char str[100 * 1024], *argv[64], *envp[1]; memset(&str, 'A', sizeof str); str[sizeof str - 1] = '\0'; for (size_t i = 0; i < sizeof argv / sizeof *argv - 1; ++i) { argv[i] = str; } argv[sizeof argv / sizeof *argv - 1] = envp[0] = 0; execve("/bin/true", argv, envp); perror("execve"); return 1; } On affected systems the program will report ARG_MAX as 10 MiB but despite that executing /bin/true with a bit over 6 MiB of command line arguments will fail with E2BIG error. Expected result is that ARG_MAX is reported as 6 MiB. Update the __sysconf function to clamp ARG_MAX value to 6 MiB if it would otherwise exceed it. This resolves bug #25305 which was market WONTFIX as suggested solution was to cap ARG_MAX at 128 KiB. As an aside and point of comparison, bionic (a libc implementation for Android systems) decided to resolve this issue by always returning 128 KiB ignoring any potential xargs regressions [2]. On older kernels this results in returning overly conservative value but that's a safer option than being aggressive and returning invalid value on recent systems. It's also worth noting that at this point all supported Linux releases have the 6 MiB barrier so only someone running an unsupported kernel version would get incorrectly truncated result. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> [1] See https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=da029c11e6b12f321f36dac8771e833b65cec962 [2] See https://android.googlesource.com/platform/bionic/+/baed51ee3a13dae4b87b11870bdf7f10bdc9efc1
* s390: Update ulpsAdhemerval Zanella2021-04-131-1/+1
| | | | | Required after 43576de04afc6 "Improve the accuracy of tgamma (BZ #26983)"
* i386: Update ulpsAdhemerval Zanella2021-04-132-4/+4
| | | | | Required after 43576de04afc6 "Improve the accuracy of tgamma (BZ #26983)"
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-09 20:39:04 +0000