about summary refs log tree commit diff
Commit message (Collapse)AuthorAgeFilesLines
* x86: Add missing IS_IN (libc) check to memmove-ssse3.SNoah Goldstein2022-06-291-16/+44
| | | | | | | | | | | | | | | | | | | | | | | Was missing to for the multiarch build rtld-memmove-ssse3.os was being built and exporting symbols: >$ nm string/rtld-memmove-ssse3.os U __GI___chk_fail 0000000000000020 T __memcpy_chk_ssse3 0000000000000040 T __memcpy_ssse3 0000000000000020 T __memmove_chk_ssse3 0000000000000040 T __memmove_ssse3 0000000000000000 T __mempcpy_chk_ssse3 0000000000000010 T __mempcpy_ssse3 U __x86_shared_cache_size_half Introduced after 2.35 in: commit 26b2478322db94edc9e0e8f577b2f71d291e5acb Author: Noah Goldstein <goldstein.w.n@gmail.com> Date: Thu Apr 14 11:47:40 2022 -0500 x86: Reduce code size of mem{move|pcpy|cpy}-ssse3
* x86-64: Properly indent X86_IFUNC_IMPL_ADD_VN argumentsH.J. Lu2022-06-291-48/+51
| | | | | | | Properly indent X86_IFUNC_IMPL_ADD_VN arguments for memchr, rawmemchr and wmemchr. Co-authored-by: H.J. Lu <hjl.tools@gmail.com>
* x86-64: Small improvements to dl-trampoline.SNoah Goldstein2022-06-292-56/+61
| | | | | | | | | | | | | | | 1. Remove sse2 instructions when using the avx512 or avx version. 2. Fixup some format nits in how the address offsets where aligned. 3. Use more space efficient instructions in the conditional AVX restoral. - vpcmpeqq -> vpcmpeqb - cmp imm32, r; jz -> inc r; jz 4. Use `rep movsb` instead of `rep movsq`. The former is guranteed to be fast with the ERMS flags, the latter is not. The latter also wastes an instruction in size setup.
* x86: Move mem{p}{mov|cpy}_{chk_}erms to its own fileNoah Goldstein2022-06-293-50/+73
| | | | | | The primary memmove_{impl}_unaligned_erms implementations don't interact with this function. Putting them in same file both wastes space and unnecessarily bloats a hot code section.
* x86: Move and slightly improve memset_ermsNoah Goldstein2022-06-293-31/+45
| | | | | | | | | | | | | | Implementation wise: 1. Remove the VZEROUPPER as memset_{impl}_unaligned_erms does not use the L(stosb) label that was previously defined. 2. Don't give the hotpath (fallthrough) to zero size. Code positioning wise: Move memset_{chk}_erms to its own file. Leaving it in between the memset_{impl}_unaligned both adds unnecessary complexity to the file and wastes space in a relatively hot cache section.
* x86: Add definition for __wmemset_chk AVX2 RTM in ifunc impl listNoah Goldstein2022-06-291-0/+4
| | | | This was simply missing and meant we weren't testing it properly.
* linux: Remove unnecessary nice.c and signal.cArjun Shankar2022-06-302-4/+0
| | | | | | | | | These files simply include the sysdeps/posix implementations which would be used even in the absence of the files. They have been unnecessary since 7b17aeda0c5e when nice and signal were removed from the syscalls.list file. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* nptl: Remove unused members from struct pthreadAdhemerval Zanella2022-06-291-7/+0
| | | | | | | It removes both pid_ununsed and cpuclock_offset_ununsed, saving about 12 bytes from struct pthread. Reviewed-by: Arjun Shankar <arjun@redhat.com>
* Linux: Forward declaration of struct iovec for process_madviseFlorian Weimer2022-06-291-5/+2
| | | | | | | | | | | | | This maintains compatibility between <sys/mman.h> and <linux/uio.h>. Before that, the addition of process_madvise made those two header files incompatible. This has been observed resulting in a build failure in LLDB's Process/Linux/NativeRegisterContextLinux_s390x.cpp source file. Fixes commit d19ee3473d68ca0e794f3a8b7677a0983ae1342e ("linux: Add process_madvise"). Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* x86: Add more feature definitions to isa-level.hNoah Goldstein2022-06-281-0/+15
| | | | | This commit doesn't change anything in itself. It is just to add definitions that will be needed by future patches.
* elf: Fix -DNDEBUG warning in _dl_start_args_adjustFlorian Weimer2022-06-281-1/+1
| | | | | This is another blocker for building glibc with the default -Werror setting and -DNDEBUG.
* elf: Fix compile error with -Werror and -DNDEBUGYang Yanchao2022-06-281-1/+2
| | | | | | | | | | | | | Using -Werror and -DNDEBUG at the same time will trigger the following compiler error: cache.c: In function 'save_cache': cache.c:758:15: error: unused variable 'old_offset' [-Werror=unused-variable] 758 | off64_t old_offset = lseek64 (fd, extension_offset, SEEK_SET); | ^~~~~~~~~~ -DNDEBUG disables the assertion, making old_offset unused. Use __attribute__ ((unused)) to disable this warning.
* x86-64: Only define used SSE/AVX/AVX512 run-time resolversH.J. Lu2022-06-273-31/+42
| | | | | | | When glibc is built with x86-64 ISA level v3, SSE run-time resolvers aren't used. For x86-64 ISA level v4 build, both SSE and AVX resolvers are unused. Check the minimum x86-64 ISA level to exclude the unused run-time resolvers.
* x86: Move CPU_FEATURE{S}_{USABLE|ARCH}_P to isa-level.hH.J. Lu2022-06-272-27/+24
| | | | | Move X86_ISA_CPU_FEATURE_USABLE_P and X86_ISA_CPU_FEATURES_ARCH_P to where MINIMUM_X86_ISA_LEVEL and XXX_X86_ISA_LEVEL are defined.
* x86: Fix backwards Prefer_No_VZEROUPPER check in ifunc-evex.hNoah Goldstein2022-06-273-26/+34
| | | | | | | | | Add third argument to X86_ISA_CPU_FEATURES_ARCH_P macro so the runtime CPU_FEATURES_ARCH_P check can be inverted if the MINIMUM_X86_ISA_LEVEL is not high enough to constantly evaluate the check. Use this new macro to correct the backwards check in ifunc-evex.h
* x86: Rename strstr_sse2 to strstr_generic as it uses string/strstr.cNoah Goldstein2022-06-273-6/+6
| | | | This is in accordance with other files in the multiarch directory.
* x86: Remove unused file wmemcmp-sse4Noah Goldstein2022-06-271-4/+0
| | | | | | | | | | | | The memcmp-sse4 was removed in: commit 7cbc03d03091d5664060924789afe46d30a5477e Author: Noah Goldstein <goldstein.w.n@gmail.com> Date: Fri Apr 15 12:28:00 2022 -0500 x86: Remove memcmp-sse4.S so this file does nothing.
* x86: Put wcs{n}len-sse4.1 in the sse4.1 text sectionNoah Goldstein2022-06-273-1/+7
| | | | | Previously was missing but the two implementations shouldn't get in the sse2 (generic) text section.
* x86: Align entry for memrchr to 64-bytes.Noah Goldstein2022-06-271-1/+1
| | | | | | | | | The function was tuned around 64-byte entry alignment and performs better for all sizes with it. As well different code boths where explicitly written to touch the minimum number of cache line i.e sizes <= 32 touch only the entry cache line.
* Makerules: Remove no-op -Wl,-d when linking libc_pic.osFangrui Song2022-06-261-6/+2
| | | | | | | | | In GNU ld, -d assigns space to common symbols for -r (i.e. change common symbols to STB_GLOBAL definitions). This option was added in commit da2d1bc5adf49352232ad0514e79fbd5dcae08e8 (1998) perhaps because ld at that time had a bug that common symbols did not override shared object definitions. -d has been long unneeded and more so since -fno-common was added to +cflags.
* m68k: optimize RTLD_STARTAndreas Schwab2022-06-251-4/+2
|
* misc: Optimize internal usage of __libc_single_threadedAdhemerval Zanella2022-06-247-4/+42
| | | | | | | | | | | | | | | By adding an internal alias to avoid the GOT indirection. On some architecture, __libc_single_thread may be accessed through copy relocations and thus it requires to update also the copies default copy. This is done by adding a new internal macro, libc_hidden_data_{proto,def}, which has an addition argument that specifies the alias name (instead of default __GI_ one). Checked on x86_64-linux-gnu and i686-linux-gnu. Reviewed-by: Fangrui Song <maskray@google.com>
* linux: Add move_mountAdhemerval Zanella2022-06-2438-5/+59
| | | | | | | | | It was added on Linux 5.2 (2db154b3ea8e14b04fee23e3fdfd5e9d17fbc6ae) as way t move a mount from one place to another and, in the next commit, allow to attach an unattached mount tree. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com>
* linux: Add fsmountAdhemerval Zanella2022-06-2438-5/+63
| | | | | | | | | | It was added on 5.2 (93766fbd2696c2c4453dd8e1070977e9cd4e6b6d) to provide a way by which a filesystem opened with fsopen and configured by a series of fsconfig calls can have a detached mount object created for it. Tested-by: Carlos O'Donell <carlos@redhat.com> Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* linux: Add fsopenAdhemerval Zanella2022-06-2439-0/+112
| | | | | | | | | It was added on Linux 5.2 (24dcb3d90a1f67fe08c68a004af37df059d74005) to start the process of preparing to create a superblock that will then be mountable, using an fd as a context handle. Tested-by: Carlos O'Donell <carlos@redhat.com> Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* resolv/tst-resolv-noaaaa: Support building for older C standardsFlorian Weimer2022-06-241-5/+9
| | | | | | | | | | | | This avoids a compilation error: tst-resolv-noaaaa.c: In function 'response': tst-resolv-noaaaa.c:74:11: error: a label can only be part of a statement and a declaration is not a statement char ipv4[4] = {192, 0, 2, i + 1}; ^~~~ tst-resolv-noaaaa.c:79:11: error: a label can only be part of a statement and a declaration is not a statement char *name = xasprintf ("ptr-%d", i); ^~~~
* resolv: Implement no-aaaa stub resolver optionFlorian Weimer2022-06-2412-12/+785
| | | | Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* support: Change non-address output format of support_format_dns_packetFlorian Weimer2022-06-242-9/+17
| | | | | | | It makes sense to include the owner name (LHS) and record type in the output, so that they can be checked for correctness. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* riscv: Use elf_machine_rela_relative to handle R_RISCV_RELATIVEKito Cheng2022-06-231-9/+9
| | | | | | | | Minor clean-up, we need to change this part in following patch, clean this up to prevent we duplicated the change twice. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
* x86: Remove faulty sanity tests for RTLD build with no multiarchNoah Goldstein2022-06-231-10/+0
| | | | | | | | | | The sanity tests where meant to ensure that the default implementation was only being built without multiarch with the exception of the multiarch/rtld-*.S files. The code used IS_IN (rtld) to check if the build for was for an multiarch/rtld-*.S file which is incorrect as IS_IN (rtld) is set for the non-multiarch build as well.
* stdlib: Fixup mbstowcs NULL __dst handling. [BZ #29279]Noah Goldstein2022-06-231-4/+4
| | | | | | | | | | | | | | | | commit 464d189b9622932a75302290625de84931656ec0 (origin/master, origin/HEAD) Author: Noah Goldstein <goldstein.w.n@gmail.com> Date: Wed Jun 22 08:24:21 2022 -0700 stdlib: Remove attr_write from mbstows if dst is NULL [BZ: 29265] Incorrectly called `__mbstowcs_chk` in the NULL __dst case which is incorrect as in the NULL __dst case we are explicitly skipping the objsize checks. As well, remove the `__always_inline` attribute which exists in `__fortify_function`. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
* x86: Replace all sse instructions with vex equivilent in avx+ filesNoah Goldstein2022-06-2275-158/+158
| | | | | | | | | | | | | Most of these don't really matter as there was no dirty upper state but we should generally avoid stray sse when its not needed. The one case that really matters is in svml_d_tanh4_core_avx2.S: blendvps %xmm0, %xmm8, %xmm7 When there was a dirty upper state. Tested on x86_64-linux
* x86: Add support for compiling {raw|w}memchr with high ISA levelNoah Goldstein2022-06-2217-604/+720
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. Refactor files so that all implementations for in the multiarch directory. - Essentially moved sse2 {raw|w}memchr.S implementation to multiarch/{raw|w}memchr-sse2.S - The non-multiarch {raw|w}memchr.S file now only includes one of the implementations in the multiarch directory based on the compiled ISA level (only used for non-multiarch builds. Otherwise we go through the ifunc selector). 2. Add ISA level build guards to different implementations. - I.e memchr-avx2.S which is ISA level 3 will only build if compiled ISA level <= 3. Otherwise there is no reason to include it as we will always use one of the ISA level 4 implementations (memchr-evex{-rtm}.S). 3. Add new multiarch/rtld-{raw}memchr.S that just include the non-multiarch {raw}memchr.S which will in turn select the best implementation based on the compiled ISA level. 4. Refactor the ifunc selector and ifunc implementation list to use the ISA level aware wrapper macros that allow functions below the compiled ISA level (with a guranteed replacement) to be skipped. - Guranteed replacement essentially means that for any ISA level build there must be a function that the baseline of the ISA supports. So for {raw|w}memchr.S since there is not ISA level 2 function, the ISA level 2 build still includes the ISA level 1 (sse2) function. Once we reach the ISA level 3 build, however, {raw|w}memchr-avx2{-rtm}.S will always be sufficient so the ISA level 1 implementation ({raw|w}memchr-sse2.S) will not be built. Tested with and without multiarch on x86_64 for ISA levels: {generic, x86-64-v2, x86-64-v3, x86-64-v4} And m32 with and without multiarch.
* x86: Add defines / utilities for making ISA specific x86 buildsNoah Goldstein2022-06-225-13/+229
| | | | | | | | | | | | | | | 1. Factor out some of the ISA level defines in isa-level.c to standalone header isa-level.h 2. Add new headers with ISA level dependent macros for handling ifuncs. Note, this file does not change any code. Tested with and without multiarch on x86_64 for ISA levels: {generic, x86-64-v2, x86-64-v3, x86-64-v4} And m32 with and without multiarch.
* stdlib: Remove attr_write from mbstows if dst is NULL [BZ: 29265]Noah Goldstein2022-06-223-5/+21
| | | | | | | | mbstows is defined if dst is NULL and is defined to special cased if dst is NULL so the fortify objsize check if incorrect in that case. Tested on x86-64 linux. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
* stdlib: Remove trailing whitespace from MakefileNoah Goldstein2022-06-221-1/+1
| | | | | This causes precommit tests to fail when pushing commits that modify this file.
* debug: make __read_chk a cancellation point (bug 29274)Andreas Schwab2022-06-223-10/+57
| | | | | The __read_chk function, as the implementation behind the fortified read function, must be a cancellation point, thus it cannot use INLINE_SYSCALL.
* s390: use LC_ALL=C for readelf callSam James2022-06-212-2/+2
| | | | | | | | | | Let's use LC_ALL=C as we do elsewhere for consistency. Tested on s390x-ibm-linux-gnu. See: 72bd208846535725ea28b8173e79ef60e57a968c Signed-off-by: Sam James <sam@gentoo.org> Reviewed-by: Stefan Liebler <stli@linux.ibm.com>
* s390: use $READELFSam James2022-06-212-2/+2
| | | | | | | | | | We already check for it in root configure.ac with AC_CHECK_TOOL. Let's use the result. Tested on s390x-ibm-linux-gnu. Signed-off-by: Sam James <sam@gentoo.org> Reviewed-by: Stefan Liebler <stli@linux.ibm.com>
* i386: Fix include paths for strspn, strcspn, and strpbrkNoah Goldstein2022-06-173-6/+6
| | | | | | | | | | | | | | commit c22eb807b0c8125101f6a274795425be2bbd0386 Author: Noah Goldstein <goldstein.w.n@gmail.com> Date: Thu Jun 16 15:07:12 2022 -0700 x86: Rename generic functions with unique postfix for clarity Changed the names of the strspn-c, strcspn-c, and strpbrk-c files in a general refactor. It didn't change the include paths for the i386 files breaking the i386 build. This commit fixes that. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com>
* elf: Silence GCC 11/12 false positive warningH.J. Lu2022-06-171-0/+10
| | | | | | | | | | Silence GCC 11/12 false positive warning with -mavx512f on dl-load.c: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106008 $ gcc -O2 -fPIC -march=x86-64 -mavx512f -S -Wall ... dl-load.c: In function ‘_dl_map_object_from_fd.constprop’: dl-load.c:1158:30: warning: ‘(((char *)loadcmds.113_68 + _933 + 16))[329406144173384849].mapend’ may be used uninitialized [-Wmaybe-uninitialized]
* x86: Rename generic functions with unique postfix for clarityNoah Goldstein2022-06-1629-76/+190
| | | | | | | | | | No functions are changed. It just renames generic implementations from '{func}_sse2' to '{func}_generic'. This is just because the postfix "_sse2" was overloaded and was used for files that had hand-optimized sse2 assembly implementations and files that just redirected back to the generic implementation. Full xcheck passed on x86_64.
* x86: Add BMI1/BMI2 checks for ISA_V3 checkNoah Goldstein2022-06-161-1/+2
| | | | | | | BMI1/BMI2 are part of the ISA V3 requirements: https://en.wikipedia.org/wiki/X86-64 And defined by GCC when building with `-march=x86-64-v3`
* x86-64: Handle fewer relocation types for RTLD_BOOTSTRAPFangrui Song2022-06-161-26/+6
| | | | | | | | | | The RTLD_BOOTSTRAP branch is used to relocate ld.so itself. It only needs to handle RELATIVE, GLOB_DAT, and JUMP_SLOT. RELATIVE has been handled (by _ELF_DYNAMIC_DO_RELOC due to DT_RELACOUNT, or RELR), so the switch statement only needs to handle GLOB_DAT and JUMP_SLOT. We can drop these `#if[n]def RTLD_BOOTSTRAP` and add a large `# ifndef RTLD_BOOTSTRAP` instead.
* aarch64: Handle fewer relocations for RTLD_BOOTSTRAPFangrui Song2022-06-151-18/+15
| | | | | | | | | The RTLD_BOOTSTRAP branch is used to relocate ld.so itself. It only needs to handle RELATIVE, GLOB_DAT, and JUMP_SLOT. TLSDESC/TLS_DTPMOD/TLS_DTPREL handling can be removed. Remove `case AARCH64_R(RELATIVE)` as well as elf_machine_rela has checked it. Tested on aarch64-linux-gnu.
* riscv: Change the relocations handled for RTLD_BOOTSTRAPFangrui Song2022-06-151-13/+10
| | | | | | | | | | | | The RTLD_BOOTSTRAP branch is used to relocate ld.so itself. It only needs to handle RELATIVE, GLOB_DAT, and the symbolic relocation type (R_RISCV_{32,64}). NONE and IRELATIVE can be removed. The code relies on ld.so having DT_RELACOUNT so that the RTLD_BOOTSTRAP branch does not need handle RELATIVE. Drop this minor size optimization for clarity. Acked-by: Palmer Dabbelt <palmer@rivosinc.com>
* x86: Cleanup bounds checking in large memcpy caseNoah Goldstein2022-06-151-8/+21
| | | | | | | | | | | | 1. Fix incorrect lower-bound threshold in L(large_memcpy_2x). Previously was using `__x86_rep_movsb_threshold` and should have been using `__x86_shared_non_temporal_threshold`. 2. Avoid reloading __x86_shared_non_temporal_threshold before the L(large_memcpy_4x) bounds check. 3. Document the second bounds check for L(large_memcpy_4x) more clearly.
* x86: Add bounds `x86_non_temporal_threshold`Noah Goldstein2022-06-152-2/+8
| | | | | | | | | | | | | | | The lower-bound (16448) and upper-bound (SIZE_MAX / 16) are assumed by memmove-vec-unaligned-erms. The lower-bound is needed because memmove-vec-unaligned-erms unrolls the loop aggressively in the L(large_memset_4x) case. The upper-bound is needed because memmove-vec-unaligned-erms right-shifts the value of `x86_non_temporal_threshold` by LOG_4X_MEMCPY_THRESH (4) which without a bound may overflow. The lack of lower-bound can be a correctness issue. The lack of upper-bound cannot.
* Remove remnant reference to ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATAFangrui Song2022-06-152-6/+2
| | | | This fixes nios2 build after commit de38b2a343e6d64b95c50004943d6107a9e380d0.
* elf: Remove ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATAFangrui Song2022-06-157-124/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | If an executable has copy relocations for extern protected data, that can only work if the library containing the definition is built with assumptions (a) the compiler emits GOT-generating relocations (b) the linker produces R_*_GLOB_DAT instead of R_*_RELATIVE. Otherwise the library uses its own definition directly and the executable accesses a stale copy. Note: the GOT relocations defeat the purpose of protected visibility as an optimization, but allow rtld to make the executable and library use the same copy when copy relocations are present, but it turns out this never worked perfectly. ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA has strange semantics when both a.so and b.so define protected var and the executable copy relocates var: b.so accesses its own copy even with GLOB_DAT. The behavior change is from commit 62da1e3b00b51383ffa7efc89d8addda0502e107 (x86) and then copied to nios2 (ae5eae7cfc9c4a8297ff82ec6b794faca1976ecc) and arc (0e7d930c4c11de896fe807f67fa1eb756c9c1e05). Without ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA, b.so accesses the copy relocated data like a.so. There is now a warning for copy relocation on protected symbol since commit 7374c02b683b7110b853a32496a619410364d70b. It's extremely unlikely anyone relies on the ELF_RTYPE_CLASS_EXTERN_PROTECTED_DATA behavior, so let's remove it: this removes a check in the symbol lookup code.
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-09 15:55:19 +0000