diff options
| author | H.J. Lu <hjl.tools@gmail.com> | 2017-07-19 06:54:03 -0700 |
|---|---|---|
| committer | H.J. Lu <hjl.tools@gmail.com> | 2017-08-04 13:29:38 -0700 |
| commit | 973da22a356ed072133562e6f1ecd833c31fa9a0 (patch) | |
| tree | 518293c48b06158f0ce466c37bf2307a888fceb8 | |
| parent | 8537e0f6cf8a1c245612fd3437789d2e16278e3b (diff) | |
| download | glibc-973da22a356ed072133562e6f1ecd833c31fa9a0.tar.gz glibc-973da22a356ed072133562e6f1ecd833c31fa9a0.tar.xz glibc-973da22a356ed072133562e6f1ecd833c31fa9a0.zip | |
i686: Implement IFUNC selectors in C
* sysdeps/i386/i686/multiarch/Makefile (sysdep_routines): Add bcopy-ia32, bzero-ia32, rawmemchr-ia32 memchr-ia32, memcmp-ia32, memcpy-ia32, memmove-ia32, mempcpy-ia32, memset-ia32, strcat-ia32, strchr-ia32, strrchr-ia32, strcpy-ia32, strcmp-ia32, strcspn-ia32, strpbrk-ia32, strspn-ia32, strlen-ia32, stpcpy-ia32, stpncpy-ia32, memcpy_chk-nonshared, mempcpy_chk-nonshared, memmove_chk-nonshared and memset_chk-nonshared * sysdeps/i386/i686/multiarch/bcopy-ia32.S: New file. * sysdeps/i386/i686/multiarch/bcopy.c: Likewise. * sysdeps/i386/i686/multiarch/bzero-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/bzero.c: Likewise. * sysdeps/i386/i686/multiarch/ifunc-memmove.h: Likewise. * sysdeps/i386/i686/multiarch/ifunc-memset.h: Likewise. * sysdeps/i386/i686/multiarch/ifunc-sse2-bsf.h: Likewise. * sysdeps/i386/i686/multiarch/ifunc-sse2-ssse3.h: Likewise. * sysdeps/i386/i686/multiarch/ifunc-sse2.h: Likewise. * sysdeps/i386/i686/multiarch/ifunc-sse4_2.h: Likewise. * sysdeps/i386/i686/multiarch/ifunc-ssse3-sse4_2.h: Likewise. * sysdeps/i386/i686/multiarch/memchr-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/memchr.c: Likewise. * sysdeps/i386/i686/multiarch/memcmp-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/memcmp.c: Likewise. * sysdeps/i386/i686/multiarch/memcpy-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/memcpy.c: Likewise. * sysdeps/i386/i686/multiarch/memcpy_chk-nonshared.S: Likewise. * sysdeps/i386/i686/multiarch/memcpy_chk.c: Likewise. * sysdeps/i386/i686/multiarch/memmove-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/memmove.c: Likewise. * sysdeps/i386/i686/multiarch/memmove_chk-nonshared.S: Likewise. * sysdeps/i386/i686/multiarch/memmove_chk.c: Likewise. * sysdeps/i386/i686/multiarch/mempcpy-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/mempcpy.c: Likewise. * sysdeps/i386/i686/multiarch/mempcpy_chk-nonshared.S: Likewise. * sysdeps/i386/i686/multiarch/mempcpy_chk.c: Likewise. * sysdeps/i386/i686/multiarch/memrchr.c: Likewise. * sysdeps/i386/i686/multiarch/memset-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/memset.c: Likewise. * sysdeps/i386/i686/multiarch/memset_chk-nonshared.S: Likewise. * sysdeps/i386/i686/multiarch/rawmemchr-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/rawmemchr.c: Likewise. * sysdeps/i386/i686/multiarch/stpcpy-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/stpcpy-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/stpcpy.c: Likewise. * sysdeps/i386/i686/multiarch/stpncpy-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/stpncpy.c: Likewise. * sysdeps/i386/i686/multiarch/strcasecmp.c: Likewise. * sysdeps/i386/i686/multiarch/strcasecmp_l.c: Likewise. * sysdeps/i386/i686/multiarch/strcat-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/strcat.c: Likewise. * sysdeps/i386/i686/multiarch/strchr-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/strchr.c: Likewise. * sysdeps/i386/i686/multiarch/strcmp-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/strcmp.c: Likewise. * sysdeps/i386/i686/multiarch/strcpy-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/strcpy.c: Likewise. * sysdeps/i386/i686/multiarch/strcspn-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/strcspn.c: Likewise. * sysdeps/i386/i686/multiarch/strlen-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/strlen.c: Likewise. * sysdeps/i386/i686/multiarch/strncase.c: Likewise. * sysdeps/i386/i686/multiarch/strncase_l.c: Likewise. * sysdeps/i386/i686/multiarch/strncat.c: Likewise. * sysdeps/i386/i686/multiarch/strncmp.c: Likewise. * sysdeps/i386/i686/multiarch/strncpy.c: Likewise. * sysdeps/i386/i686/multiarch/strnlen.c: Likewise. * sysdeps/i386/i686/multiarch/strpbrk-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/strpbrk.c: Likewise. * sysdeps/i386/i686/multiarch/strrchr-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/strrchr.c: Likewise. * sysdeps/i386/i686/multiarch/strspn-ia32.S: Likewise. * sysdeps/i386/i686/multiarch/strspn.c: Likewise. * sysdeps/i386/i686/multiarch/wcschr.c: Likewise. * sysdeps/i386/i686/multiarch/wcscmp.c: Likewise. * sysdeps/i386/i686/multiarch/wcscpy.c: Likewise. * sysdeps/i386/i686/multiarch/wcslen.c: Likewise. * sysdeps/i386/i686/multiarch/wcsrchr.c: Likewise. * sysdeps/i386/i686/multiarch/wmemcmp.c: Likewise. * sysdeps/i386/i686/multiarch/bcopy.S: Removed. * sysdeps/i386/i686/multiarch/bzero.S: Likewise. * sysdeps/i386/i686/multiarch/memchr.S: Likewise. * sysdeps/i386/i686/multiarch/memcmp.S: Likewise. * sysdeps/i386/i686/multiarch/memcpy.S: Likewise. * sysdeps/i386/i686/multiarch/memcpy_chk.S: Likewise. * sysdeps/i386/i686/multiarch/memmove.S: Likewise. * sysdeps/i386/i686/multiarch/memmove_chk.S: Likewise. * sysdeps/i386/i686/multiarch/mempcpy.S: Likewise. * sysdeps/i386/i686/multiarch/mempcpy_chk.S: Likewise. * sysdeps/i386/i686/multiarch/memrchr.S: Likewise. * sysdeps/i386/i686/multiarch/memset.S: Likewise. * sysdeps/i386/i686/multiarch/memset_chk.S: Likewise. * sysdeps/i386/i686/multiarch/rawmemchr.S: Likewise. * sysdeps/i386/i686/multiarch/stpcpy.S: Likewise. * sysdeps/i386/i686/multiarch/stpncpy.S: Likewise. * sysdeps/i386/i686/multiarch/strcasecmp.S: Likewise. * sysdeps/i386/i686/multiarch/strcasecmp_l.S: Likewise. * sysdeps/i386/i686/multiarch/strcat.S: Likewise. * sysdeps/i386/i686/multiarch/strcmp.S: Likewise. * sysdeps/i386/i686/multiarch/strcpy.S: Likewise. * sysdeps/i386/i686/multiarch/strcspn.S: Likewise. * sysdeps/i386/i686/multiarch/strlen.S: Likewise. * sysdeps/i386/i686/multiarch/strncase.S: Likewise. * sysdeps/i386/i686/multiarch/strncase_l.S: Likewise. * sysdeps/i386/i686/multiarch/strncat.S: Likewise. * sysdeps/i386/i686/multiarch/strncmp.S: Likewise. * sysdeps/i386/i686/multiarch/strncpy.S: Likewise. * sysdeps/i386/i686/multiarch/strnlen.S: Likewise. * sysdeps/i386/i686/multiarch/strpbrk.S: Likewise. * sysdeps/i386/i686/multiarch/strrchr.S: Likewise. * sysdeps/i386/i686/multiarch/strspn.S: Likewise. * sysdeps/i386/i686/multiarch/wcschr.S: Likewise. * sysdeps/i386/i686/multiarch/wcscmp.S: Likewise. * sysdeps/i386/i686/multiarch/wcscpy.S: Likewise. * sysdeps/i386/i686/multiarch/wcslen.S: Likewise. * sysdeps/i386/i686/multiarch/wcsrchr.S: Likewise. * sysdeps/i386/i686/multiarch/wmemcmp.S: Likewise.
100 files changed, 2197 insertions, 1732 deletions
diff --git a/ChangeLog b/ChangeLog index 268436266c..db25c7fa11 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,124 @@ 2017-08-04 H.J. Lu <hongjiu.lu@intel.com> + * sysdeps/i386/i686/multiarch/Makefile (sysdep_routines): Add + bcopy-ia32, bzero-ia32, rawmemchr-ia32 memchr-ia32, + memcmp-ia32, memcpy-ia32, memmove-ia32, mempcpy-ia32, + memset-ia32, strcat-ia32, strchr-ia32, strrchr-ia32, + strcpy-ia32, strcmp-ia32, strcspn-ia32, strpbrk-ia32, + strspn-ia32, strlen-ia32, stpcpy-ia32, stpncpy-ia32, + memcpy_chk-nonshared, mempcpy_chk-nonshared, + memmove_chk-nonshared and memset_chk-nonshared + * sysdeps/i386/i686/multiarch/bcopy-ia32.S: New file. + * sysdeps/i386/i686/multiarch/bcopy.c: Likewise. + * sysdeps/i386/i686/multiarch/bzero-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/bzero.c: Likewise. + * sysdeps/i386/i686/multiarch/ifunc-memmove.h: Likewise. + * sysdeps/i386/i686/multiarch/ifunc-memset.h: Likewise. + * sysdeps/i386/i686/multiarch/ifunc-sse2-bsf.h: Likewise. + * sysdeps/i386/i686/multiarch/ifunc-sse2-ssse3.h: Likewise. + * sysdeps/i386/i686/multiarch/ifunc-sse2.h: Likewise. + * sysdeps/i386/i686/multiarch/ifunc-sse4_2.h: Likewise. + * sysdeps/i386/i686/multiarch/ifunc-ssse3-sse4_2.h: Likewise. + * sysdeps/i386/i686/multiarch/memchr-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/memchr.c: Likewise. + * sysdeps/i386/i686/multiarch/memcmp-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/memcmp.c: Likewise. + * sysdeps/i386/i686/multiarch/memcpy-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/memcpy.c: Likewise. + * sysdeps/i386/i686/multiarch/memcpy_chk-nonshared.S: Likewise. + * sysdeps/i386/i686/multiarch/memcpy_chk.c: Likewise. + * sysdeps/i386/i686/multiarch/memmove-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/memmove.c: Likewise. + * sysdeps/i386/i686/multiarch/memmove_chk-nonshared.S: Likewise. + * sysdeps/i386/i686/multiarch/memmove_chk.c: Likewise. + * sysdeps/i386/i686/multiarch/mempcpy-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/mempcpy.c: Likewise. + * sysdeps/i386/i686/multiarch/mempcpy_chk-nonshared.S: Likewise. + * sysdeps/i386/i686/multiarch/mempcpy_chk.c: Likewise. + * sysdeps/i386/i686/multiarch/memrchr.c: Likewise. + * sysdeps/i386/i686/multiarch/memset-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/memset.c: Likewise. + * sysdeps/i386/i686/multiarch/memset_chk-nonshared.S: Likewise. + * sysdeps/i386/i686/multiarch/rawmemchr-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/rawmemchr.c: Likewise. + * sysdeps/i386/i686/multiarch/stpcpy-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/stpcpy-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/stpcpy.c: Likewise. + * sysdeps/i386/i686/multiarch/stpncpy-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/stpncpy.c: Likewise. + * sysdeps/i386/i686/multiarch/strcasecmp.c: Likewise. + * sysdeps/i386/i686/multiarch/strcasecmp_l.c: Likewise. + * sysdeps/i386/i686/multiarch/strcat-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/strcat.c: Likewise. + * sysdeps/i386/i686/multiarch/strchr-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/strchr.c: Likewise. + * sysdeps/i386/i686/multiarch/strcmp-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/strcmp.c: Likewise. + * sysdeps/i386/i686/multiarch/strcpy-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/strcpy.c: Likewise. + * sysdeps/i386/i686/multiarch/strcspn-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/strcspn.c: Likewise. + * sysdeps/i386/i686/multiarch/strlen-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/strlen.c: Likewise. + * sysdeps/i386/i686/multiarch/strncase.c: Likewise. + * sysdeps/i386/i686/multiarch/strncase_l.c: Likewise. + * sysdeps/i386/i686/multiarch/strncat.c: Likewise. + * sysdeps/i386/i686/multiarch/strncmp.c: Likewise. + * sysdeps/i386/i686/multiarch/strncpy.c: Likewise. + * sysdeps/i386/i686/multiarch/strnlen.c: Likewise. + * sysdeps/i386/i686/multiarch/strpbrk-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/strpbrk.c: Likewise. + * sysdeps/i386/i686/multiarch/strrchr-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/strrchr.c: Likewise. + * sysdeps/i386/i686/multiarch/strspn-ia32.S: Likewise. + * sysdeps/i386/i686/multiarch/strspn.c: Likewise. + * sysdeps/i386/i686/multiarch/wcschr.c: Likewise. + * sysdeps/i386/i686/multiarch/wcscmp.c: Likewise. + * sysdeps/i386/i686/multiarch/wcscpy.c: Likewise. + * sysdeps/i386/i686/multiarch/wcslen.c: Likewise. + * sysdeps/i386/i686/multiarch/wcsrchr.c: Likewise. + * sysdeps/i386/i686/multiarch/wmemcmp.c: Likewise. + * sysdeps/i386/i686/multiarch/bcopy.S: Removed. + * sysdeps/i386/i686/multiarch/bzero.S: Likewise. + * sysdeps/i386/i686/multiarch/memchr.S: Likewise. + * sysdeps/i386/i686/multiarch/memcmp.S: Likewise. + * sysdeps/i386/i686/multiarch/memcpy.S: Likewise. + * sysdeps/i386/i686/multiarch/memcpy_chk.S: Likewise. + * sysdeps/i386/i686/multiarch/memmove.S: Likewise. + * sysdeps/i386/i686/multiarch/memmove_chk.S: Likewise. + * sysdeps/i386/i686/multiarch/mempcpy.S: Likewise. + * sysdeps/i386/i686/multiarch/mempcpy_chk.S: Likewise. + * sysdeps/i386/i686/multiarch/memrchr.S: Likewise. + * sysdeps/i386/i686/multiarch/memset.S: Likewise. + * sysdeps/i386/i686/multiarch/memset_chk.S: Likewise. + * sysdeps/i386/i686/multiarch/rawmemchr.S: Likewise. + * sysdeps/i386/i686/multiarch/stpcpy.S: Likewise. + * sysdeps/i386/i686/multiarch/stpncpy.S: Likewise. + * sysdeps/i386/i686/multiarch/strcasecmp.S: Likewise. + * sysdeps/i386/i686/multiarch/strcasecmp_l.S: Likewise. + * sysdeps/i386/i686/multiarch/strcat.S: Likewise. + * sysdeps/i386/i686/multiarch/strcmp.S: Likewise. + * sysdeps/i386/i686/multiarch/strcpy.S: Likewise. + * sysdeps/i386/i686/multiarch/strcspn.S: Likewise. + * sysdeps/i386/i686/multiarch/strlen.S: Likewise. + * sysdeps/i386/i686/multiarch/strncase.S: Likewise. + * sysdeps/i386/i686/multiarch/strncase_l.S: Likewise. + * sysdeps/i386/i686/multiarch/strncat.S: Likewise. + * sysdeps/i386/i686/multiarch/strncmp.S: Likewise. + * sysdeps/i386/i686/multiarch/strncpy.S: Likewise. + * sysdeps/i386/i686/multiarch/strnlen.S: Likewise. + * sysdeps/i386/i686/multiarch/strpbrk.S: Likewise. + * sysdeps/i386/i686/multiarch/strrchr.S: Likewise. + * sysdeps/i386/i686/multiarch/strspn.S: Likewise. + * sysdeps/i386/i686/multiarch/wcschr.S: Likewise. + * sysdeps/i386/i686/multiarch/wcscmp.S: Likewise. + * sysdeps/i386/i686/multiarch/wcscpy.S: Likewise. + * sysdeps/i386/i686/multiarch/wcslen.S: Likewise. + * sysdeps/i386/i686/multiarch/wcsrchr.S: Likewise. + * sysdeps/i386/i686/multiarch/wmemcmp.S: Likewise. + +2017-08-04 H.J. Lu <hongjiu.lu@intel.com> + * sysdeps/x86_64/fpu/multiarch/Makefile (libmvec-sysdep_routines) Add svml_d_cos2_core-sse2, svml_d_cos4_core-sse, svml_d_cos8_core-avx2, svml_d_exp2_core-sse2, diff --git a/sysdeps/i386/i686/multiarch/Makefile b/sysdeps/i386/i686/multiarch/Makefile index 4a0c20c051..bf75a9947f 100644 --- a/sysdeps/i386/i686/multiarch/Makefile +++ b/sysdeps/i386/i686/multiarch/Makefile @@ -24,7 +24,13 @@ sysdep_routines += bzero-sse2 memset-sse2 memcpy-ssse3 mempcpy-ssse3 \ strcasecmp_l-sse4 strncase_l-sse4 \ bcopy-sse2-unaligned memcpy-sse2-unaligned \ mempcpy-sse2-unaligned memmove-sse2-unaligned \ - strcspn-c strpbrk-c strspn-c + strcspn-c strpbrk-c strspn-c \ + bcopy-ia32 bzero-ia32 rawmemchr-ia32 \ + memchr-ia32 memcmp-ia32 memcpy-ia32 memmove-ia32 \ + mempcpy-ia32 memset-ia32 strcat-ia32 strchr-ia32 \ + strrchr-ia32 strcpy-ia32 strcmp-ia32 strcspn-ia32 \ + strpbrk-ia32 strspn-ia32 strlen-ia32 stpcpy-ia32 \ + stpncpy-ia32 CFLAGS-varshift.c += -msse4 CFLAGS-strcspn-c.c += -msse4 CFLAGS-strpbrk-c.c += -msse4 @@ -42,3 +48,8 @@ libm-sysdep_routines += s_fma-fma s_fmaf-fma CFLAGS-s_fma-fma.c += -mavx -mfpmath=sse CFLAGS-s_fmaf-fma.c += -mavx -mfpmath=sse endif + +ifeq ($(subdir),debug) +sysdep_routines += memcpy_chk-nonshared mempcpy_chk-nonshared \ + memmove_chk-nonshared memset_chk-nonshared +endif diff --git a/sysdeps/i386/i686/multiarch/bcopy-ia32.S b/sysdeps/i386/i686/multiarch/bcopy-ia32.S new file mode 100644 index 0000000000..606cd2edf8 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/bcopy-ia32.S @@ -0,0 +1,20 @@ +/* bcopy optimized for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#define bcopy __bcopy_ia32 +#include <sysdeps/i386/i686/bcopy.S> diff --git a/sysdeps/i386/i686/multiarch/bcopy.S b/sysdeps/i386/i686/multiarch/bcopy.S deleted file mode 100644 index 877f82c28f..0000000000 --- a/sysdeps/i386/i686/multiarch/bcopy.S +++ /dev/null @@ -1,59 +0,0 @@ -/* Multiple versions of bcopy - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in lib. */ -#if IS_IN (libc) - .text -ENTRY(bcopy) - .type bcopy, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__bcopy_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__bcopy_sse2_unaligned) - HAS_ARCH_FEATURE (Fast_Unaligned_Load) - jnz 2f - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__bcopy_ssse3) - HAS_ARCH_FEATURE (Fast_Rep_String) - jz 2f - LOAD_FUNC_GOT_EAX (__bcopy_ssse3_rep) -2: ret -END(bcopy) - -# undef ENTRY -# define ENTRY(name) \ - .type __bcopy_ia32, @function; \ - .p2align 4; \ - .globl __bcopy_ia32; \ - .hidden __bcopy_ia32; \ - __bcopy_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __bcopy_ia32, .-__bcopy_ia32 - -#endif - -#include "../bcopy.S" diff --git a/sysdeps/i386/i686/multiarch/wcsrchr.S b/sysdeps/i386/i686/multiarch/bcopy.c index cf67333995..70cd665c10 100644 --- a/sysdeps/i386/i686/multiarch/wcsrchr.S +++ b/sysdeps/i386/i686/multiarch/bcopy.c @@ -1,7 +1,6 @@ -/* Multiple versions of wcsrchr +/* Multiple versions of bcopy. All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. + Copyright (C) 2017 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -18,18 +17,14 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <sysdep.h> -#include <init-arch.h> - +/* Define multiple versions only for the definition in libc. */ #if IS_IN (libc) - .text -ENTRY(wcsrchr) - .type wcsrchr, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__wcsrchr_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__wcsrchr_sse2) -2: ret -END(wcsrchr) +# define bcopy __redirect_bcopy +# include <string.h> +# undef bcopy + +# define SYMBOL_NAME bcopy +# include "ifunc-memmove.h" + +libc_ifunc_redirected (__redirect_bcopy, bcopy, IFUNC_SELECTOR ()); #endif diff --git a/sysdeps/i386/i686/multiarch/bzero-ia32.S b/sysdeps/i386/i686/multiarch/bzero-ia32.S new file mode 100644 index 0000000000..8df871f5f8 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/bzero-ia32.S @@ -0,0 +1,38 @@ +/* bzero optimized for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <sysdep.h> +#include <init-arch.h> + +#if IS_IN (libc) +# define __bzero __bzero_ia32 + +# ifdef SHARED +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI___bzero; __GI___bzero = __bzero +# endif + +# undef weak_alias +# define weak_alias(original, alias) + +# include <sysdeps/i386/i686/bzero.S> +#endif diff --git a/sysdeps/i386/i686/multiarch/bzero.S b/sysdeps/i386/i686/multiarch/bzero.S deleted file mode 100644 index 9dac490aa2..0000000000 --- a/sysdeps/i386/i686/multiarch/bzero.S +++ /dev/null @@ -1,62 +0,0 @@ -/* Multiple versions of bzero - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in lib. */ -#if IS_IN (libc) - .text -ENTRY(__bzero) - .type __bzero, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__bzero_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX ( __bzero_sse2) - HAS_ARCH_FEATURE (Fast_Rep_String) - jz 2f - LOAD_FUNC_GOT_EAX (__bzero_sse2_rep) -2: ret -END(__bzero) - -# undef ENTRY -# define ENTRY(name) \ - .type __bzero_ia32, @function; \ - .p2align 4; \ - .globl __bzero_ia32; \ - .hidden __bzero_ia32; \ - __bzero_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __bzero_ia32, .-__bzero_ia32 - -# ifdef SHARED -# undef libc_hidden_builtin_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI___bzero; __GI___bzero = __bzero_ia32 -# endif -#endif - -#include "../bzero.S" diff --git a/sysdeps/i386/i686/multiarch/wcschr.S b/sysdeps/i386/i686/multiarch/bzero.c index d3c65a6436..0b029fda70 100644 --- a/sysdeps/i386/i686/multiarch/wcschr.S +++ b/sysdeps/i386/i686/multiarch/bzero.c @@ -1,7 +1,6 @@ -/* Multiple versions of wcschr +/* Multiple versions of bzero. All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. + Copyright (C) 2017 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -18,19 +17,16 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <sysdep.h> -#include <init-arch.h> - +/* Define multiple versions only for the definition in libc. */ #if IS_IN (libc) - .text -ENTRY(__wcschr) - .type wcschr, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__wcschr_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__wcschr_sse2) -2: ret -END(__wcschr) -weak_alias (__wcschr, wcschr) +# define bzero __redirect_bzero +# include <string.h> +# undef bzero + +# define SYMBOL_NAME bzero +# include "ifunc-memset.h" + +libc_ifunc_redirected (__redirect_bzero, __bzero, IFUNC_SELECTOR ()); + +weak_alias (__bzero, bzero) #endif diff --git a/sysdeps/i386/i686/multiarch/ifunc-memmove.h b/sysdeps/i386/i686/multiarch/ifunc-memmove.h new file mode 100644 index 0000000000..9c71414480 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/ifunc-memmove.h @@ -0,0 +1,45 @@ +/* Common definition for memmove/memmove_chk ifunc selections. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <init-arch.h> + +extern __typeof (REDIRECT_NAME) OPTIMIZE (ia32) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2_unaligned) + attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (ssse3) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (ssse3_rep) attribute_hidden; + +static inline void * +IFUNC_SELECTOR (void) +{ + const struct cpu_features* cpu_features = __get_cpu_features (); + + if (CPU_FEATURES_ARCH_P (cpu_features, Fast_Unaligned_Load)) + return OPTIMIZE (sse2_unaligned); + + if (CPU_FEATURES_CPU_P (cpu_features, SSSE3)) + { + if (CPU_FEATURES_ARCH_P (cpu_features, Fast_Rep_String)) + return OPTIMIZE (ssse3_rep); + + return OPTIMIZE (ssse3); + } + + return OPTIMIZE (ia32); +} diff --git a/sysdeps/i386/i686/multiarch/ifunc-memset.h b/sysdeps/i386/i686/multiarch/ifunc-memset.h new file mode 100644 index 0000000000..77933fdbc1 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/ifunc-memset.h @@ -0,0 +1,40 @@ +/* Common definition for memset/memset_chk ifunc selections. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <init-arch.h> + +extern __typeof (REDIRECT_NAME) OPTIMIZE (ia32) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2_rep) attribute_hidden; + +static inline void * +IFUNC_SELECTOR (void) +{ + const struct cpu_features* cpu_features = __get_cpu_features (); + + if (CPU_FEATURES_CPU_P (cpu_features, SSE2)) + { + if (CPU_FEATURES_ARCH_P (cpu_features, Fast_Rep_String)) + return OPTIMIZE (sse2_rep); + + return OPTIMIZE (sse2); + } + + return OPTIMIZE (ia32); +} diff --git a/sysdeps/i386/i686/multiarch/ifunc-sse2-bsf.h b/sysdeps/i386/i686/multiarch/ifunc-sse2-bsf.h new file mode 100644 index 0000000000..0b9b99d02e --- /dev/null +++ b/sysdeps/i386/i686/multiarch/ifunc-sse2-bsf.h @@ -0,0 +1,40 @@ +/* Common definition for ifunc selections optimized with SSE2 and BSF. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <init-arch.h> + +extern __typeof (REDIRECT_NAME) OPTIMIZE (ia32) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2_bsf) attribute_hidden; + +static inline void * +IFUNC_SELECTOR (void) +{ + const struct cpu_features* cpu_features = __get_cpu_features (); + + if (CPU_FEATURES_CPU_P (cpu_features, SSE2)) + { + if (CPU_FEATURES_ARCH_P (cpu_features, Slow_BSF)) + return OPTIMIZE (sse2); + + return OPTIMIZE (sse2_bsf); + } + + return OPTIMIZE (ia32); +} diff --git a/sysdeps/i386/i686/multiarch/ifunc-sse2-ssse3.h b/sysdeps/i386/i686/multiarch/ifunc-sse2-ssse3.h new file mode 100644 index 0000000000..a4c3645ad2 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/ifunc-sse2-ssse3.h @@ -0,0 +1,40 @@ +/* Common definition for ifunc selections optimized with SSE2 and + SSSE3. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <init-arch.h> + +extern __typeof (REDIRECT_NAME) OPTIMIZE (ia32) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (ssse3) attribute_hidden; + +static inline void * +IFUNC_SELECTOR (void) +{ + const struct cpu_features* cpu_features = __get_cpu_features (); + + if (CPU_FEATURES_CPU_P (cpu_features, SSE2) + && CPU_FEATURES_ARCH_P (cpu_features, Fast_Rep_String)) + return OPTIMIZE (sse2); + + if (CPU_FEATURES_CPU_P (cpu_features, SSSE3)) + return OPTIMIZE (ssse3); + + return OPTIMIZE (ia32); +} diff --git a/sysdeps/i386/i686/multiarch/ifunc-sse2.h b/sysdeps/i386/i686/multiarch/ifunc-sse2.h new file mode 100644 index 0000000000..f4868aae06 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/ifunc-sse2.h @@ -0,0 +1,34 @@ +/* Common definition for ifunc selections optimized with SSE2. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <init-arch.h> + +extern __typeof (REDIRECT_NAME) OPTIMIZE (ia32) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2) attribute_hidden; + +static inline void * +IFUNC_SELECTOR (void) +{ + const struct cpu_features* cpu_features = __get_cpu_features (); + + if (CPU_FEATURES_CPU_P (cpu_features, SSE2)) + return OPTIMIZE (sse2); + + return OPTIMIZE (ia32); +} diff --git a/sysdeps/i386/i686/multiarch/ifunc-sse4_2.h b/sysdeps/i386/i686/multiarch/ifunc-sse4_2.h new file mode 100644 index 0000000000..d4f883a77b --- /dev/null +++ b/sysdeps/i386/i686/multiarch/ifunc-sse4_2.h @@ -0,0 +1,34 @@ +/* Common definition for ifunc selections optimized with SSE4_2. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <init-arch.h> + +extern __typeof (REDIRECT_NAME) OPTIMIZE (ia32) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (sse42) attribute_hidden; + +static inline void * +IFUNC_SELECTOR (void) +{ + const struct cpu_features* cpu_features = __get_cpu_features (); + + if (CPU_FEATURES_CPU_P (cpu_features, SSE4_2)) + return OPTIMIZE (sse42); + + return OPTIMIZE (ia32); +} diff --git a/sysdeps/i386/i686/multiarch/ifunc-ssse3-sse4_2.h b/sysdeps/i386/i686/multiarch/ifunc-ssse3-sse4_2.h new file mode 100644 index 0000000000..e19d413237 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/ifunc-ssse3-sse4_2.h @@ -0,0 +1,39 @@ +/* Common definition for ifunc selections optimized with SSSE3 and + SSE4_2. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#include <init-arch.h> + +extern __typeof (REDIRECT_NAME) OPTIMIZE (ia32) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (ssse3) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (sse4_2) attribute_hidden; + +static inline void * +IFUNC_SELECTOR (void) +{ + const struct cpu_features* cpu_features = __get_cpu_features (); + + if (CPU_FEATURES_CPU_P (cpu_features, SSE4_2)) + return OPTIMIZE (sse4_2); + + if (CPU_FEATURES_CPU_P (cpu_features, SSSE3)) + return OPTIMIZE (ssse3); + + return OPTIMIZE (ia32); +} diff --git a/sysdeps/i386/i686/multiarch/memchr-ia32.S b/sysdeps/i386/i686/multiarch/memchr-ia32.S new file mode 100644 index 0000000000..26638210f2 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memchr-ia32.S @@ -0,0 +1,35 @@ +/* memchr optimized for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define __memchr __memchr_ia32 + +# ifdef SHARED +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_memchr; __GI_memchr = __memchr +# endif + +# undef weak_alias +# define weak_alias(original, alias) +#endif + +#include <sysdeps/i386/memchr.S> diff --git a/sysdeps/i386/i686/multiarch/memchr.S b/sysdeps/i386/i686/multiarch/memchr.S deleted file mode 100644 index bd0dace290..0000000000 --- a/sysdeps/i386/i686/multiarch/memchr.S +++ /dev/null @@ -1,65 +0,0 @@ -/* Multiple versions of memchr - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -#if IS_IN (libc) - .text -ENTRY(__memchr) - .type __memchr, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - HAS_CPU_FEATURE (SSE2) - jz 2f - HAS_ARCH_FEATURE (Slow_BSF) - jz 3f - - LOAD_FUNC_GOT_EAX ( __memchr_sse2) - ret - -2: LOAD_FUNC_GOT_EAX (__memchr_ia32) - ret - -3: LOAD_FUNC_GOT_EAX (__memchr_sse2_bsf) - ret -END(__memchr) - -weak_alias(__memchr, memchr) - -# undef ENTRY -# define ENTRY(name) \ - .type __memchr_ia32, @function; \ - .globl __memchr_ia32; \ - .p2align 4; \ - __memchr_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __memchr_ia32, .-__memchr_ia32 - -# undef libc_hidden_builtin_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI_memchr; __GI_memchr = __memchr_ia32 - -#endif -#include "../../memchr.S" diff --git a/sysdeps/i386/i686/multiarch/wcslen.S b/sysdeps/i386/i686/multiarch/memchr.c index 6ef9b6e7b5..f7cbb1441d 100644 --- a/sysdeps/i386/i686/multiarch/wcslen.S +++ b/sysdeps/i386/i686/multiarch/memchr.c @@ -1,7 +1,6 @@ -/* Multiple versions of wcslen +/* Multiple versions of memchr. All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. + Copyright (C) 2017 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -18,20 +17,16 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <sysdep.h> -#include <init-arch.h> - +/* Define multiple versions only for the definition in libc. */ #if IS_IN (libc) - .text -ENTRY(__wcslen) - .type __wcslen, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__wcslen_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__wcslen_sse2) -2: ret -END(__wcslen) +# define memchr __redirect_memchr +# include <string.h> +# undef memchr + +# define SYMBOL_NAME memchr +# include "ifunc-sse2-bsf.h" + +libc_ifunc_redirected (__redirect_memchr, __memchr, IFUNC_SELECTOR ()); -weak_alias(__wcslen, wcslen) +weak_alias (__memchr, memchr) #endif diff --git a/sysdeps/i386/i686/multiarch/memcmp-ia32.S b/sysdeps/i386/i686/multiarch/memcmp-ia32.S new file mode 100644 index 0000000000..737a9145dc --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memcmp-ia32.S @@ -0,0 +1,35 @@ +/* memcmp optimized for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define memcmp __memcmp_ia32 + +# ifdef SHARED +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_memcmp; __GI_memcmp = memcmp +# endif + +# undef weak_alias +# define weak_alias(original, alias) +#endif + +#include <sysdeps/i386/i686/memcmp.S> diff --git a/sysdeps/i386/i686/multiarch/memcmp.S b/sysdeps/i386/i686/multiarch/memcmp.S deleted file mode 100644 index 1fc5994a17..0000000000 --- a/sysdeps/i386/i686/multiarch/memcmp.S +++ /dev/null @@ -1,62 +0,0 @@ -/* Multiple versions of memcmp - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in libc. */ -#if IS_IN (libc) - .text -ENTRY(memcmp) - .type memcmp, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__memcmp_ia32) - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__memcmp_ssse3) - HAS_CPU_FEATURE (SSE4_2) - jz 2f - LOAD_FUNC_GOT_EAX (__memcmp_sse4_2) -2: ret -END(memcmp) - -# undef ENTRY -# define ENTRY(name) \ - .type __memcmp_ia32, @function; \ - .p2align 4; \ - .globl __memcmp_ia32; \ - .hidden __memcmp_ia32; \ - __memcmp_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __memcmp_ia32, .-__memcmp_ia32 - -# ifdef SHARED -# undef libc_hidden_builtin_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI_memcmp; __GI_memcmp = __memcmp_ia32 -# endif -#endif - -#include "../memcmp.S" diff --git a/sysdeps/i386/i686/multiarch/strnlen.S b/sysdeps/i386/i686/multiarch/memcmp.c index d241522c70..5763c94898 100644 --- a/sysdeps/i386/i686/multiarch/strnlen.S +++ b/sysdeps/i386/i686/multiarch/memcmp.c @@ -1,7 +1,6 @@ -/* Multiple versions of strnlen +/* Multiple versions of memcmp. All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. + Copyright (C) 2017 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -18,20 +17,16 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <sysdep.h> -#include <init-arch.h> - +/* Define multiple versions only for the definition in libc. */ #if IS_IN (libc) - .text -ENTRY(__strnlen) - .type __strnlen, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__strnlen_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__strnlen_sse2) -2: ret -END(__strnlen) +# define memcmp __redirect_memcmp +# include <string.h> +# undef memcmp + +# define SYMBOL_NAME memcmp +# include "ifunc-ssse3-sse4_2.h" + +libc_ifunc_redirected (__redirect_memcmp, memcmp, IFUNC_SELECTOR ()); -weak_alias(__strnlen, strnlen) +weak_alias (memcmp, bcmp) #endif diff --git a/sysdeps/i386/i686/multiarch/memcpy-ia32.S b/sysdeps/i386/i686/multiarch/memcpy-ia32.S new file mode 100644 index 0000000000..51ec4d43cc --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memcpy-ia32.S @@ -0,0 +1,31 @@ +/* memcpy optimized for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined SHARED && IS_IN (libc) +# define memcpy __memcpy_ia32 +# define __memcpy_chk __memcpy_chk_ia32 + +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_memcpy; __GI_memcpy = memcpy +#endif + +#include <sysdeps/i386/i686/memcpy.S> diff --git a/sysdeps/i386/i686/multiarch/memcpy.S b/sysdeps/i386/i686/multiarch/memcpy.S deleted file mode 100644 index f725944620..0000000000 --- a/sysdeps/i386/i686/multiarch/memcpy.S +++ /dev/null @@ -1,78 +0,0 @@ -/* Multiple versions of memcpy - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in lib and for - DSO. In static binaries we need memcpy before the initialization - happened. */ -#if defined SHARED && IS_IN (libc) - .text -ENTRY(memcpy) - .type memcpy, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__memcpy_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__memcpy_sse2_unaligned) - HAS_ARCH_FEATURE (Fast_Unaligned_Load) - jnz 2f - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__memcpy_ssse3) - HAS_ARCH_FEATURE (Fast_Rep_String) - jz 2f - LOAD_FUNC_GOT_EAX (__memcpy_ssse3_rep) -2: ret -END(memcpy) - -# undef ENTRY -# define ENTRY(name) \ - .type __memcpy_ia32, @function; \ - .p2align 4; \ - .globl __memcpy_ia32; \ - .hidden __memcpy_ia32; \ - __memcpy_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __memcpy_ia32, .-__memcpy_ia32 - -# undef ENTRY_CHK -# define ENTRY_CHK(name) \ - .type __memcpy_chk_ia32, @function; \ - .globl __memcpy_chk_ia32; \ - .p2align 4; \ - __memcpy_chk_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END_CHK -# define END_CHK(name) \ - cfi_endproc; .size __memcpy_chk_ia32, .-__memcpy_chk_ia32 - -# undef libc_hidden_builtin_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI_memcpy; __GI_memcpy = __memcpy_ia32 -#endif - -#include "../memcpy.S" diff --git a/sysdeps/i386/i686/multiarch/memcpy.c b/sysdeps/i386/i686/multiarch/memcpy.c new file mode 100644 index 0000000000..33791223b2 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memcpy.c @@ -0,0 +1,32 @@ +/* Multiple versions of memcpy. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in lib and for + DSO. In static binaries we need memcpy before the initialization + happened. */ +#if defined SHARED && IS_IN (libc) +# define memcpy __redirect_memcpy +# include <string.h> +# undef memcpy + +# define SYMBOL_NAME memcpy +# include "ifunc-memmove.h" + +libc_ifunc_redirected (__redirect_memcpy, memcpy, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/memcpy_chk-nonshared.S b/sysdeps/i386/i686/multiarch/memcpy_chk-nonshared.S new file mode 100644 index 0000000000..91c01f7326 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memcpy_chk-nonshared.S @@ -0,0 +1,21 @@ +/* Non-shared version of memcpy_chk for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) && !defined SHARED +# include <sysdeps/i386/memcpy_chk.S> +#endif diff --git a/sysdeps/i386/i686/multiarch/memcpy_chk.S b/sysdeps/i386/i686/multiarch/memcpy_chk.S deleted file mode 100644 index 1b4fbe2e6f..0000000000 --- a/sysdeps/i386/i686/multiarch/memcpy_chk.S +++ /dev/null @@ -1,50 +0,0 @@ -/* Multiple versions of __memcpy_chk - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in lib and for - DSO. There are no multiarch memcpy functions for static binaries. - */ -#if IS_IN (libc) -# ifdef SHARED - .text -ENTRY(__memcpy_chk) - .type __memcpy_chk, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__memcpy_chk_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__memcpy_chk_sse2_unaligned) - HAS_ARCH_FEATURE (Fast_Unaligned_Load) - jnz 2f - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__memcpy_chk_ssse3) - HAS_ARCH_FEATURE (Fast_Rep_String) - jz 2f - LOAD_FUNC_GOT_EAX (__memcpy_chk_ssse3_rep) -2: ret -END(__memcpy_chk) -# else -# include "../memcpy_chk.S" -# endif -#endif diff --git a/sysdeps/i386/i686/multiarch/memcpy_chk.c b/sysdeps/i386/i686/multiarch/memcpy_chk.c new file mode 100644 index 0000000000..6347c21e3c --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memcpy_chk.c @@ -0,0 +1,31 @@ +/* Multiple versions of __memcpy_chk + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc.so. */ +#if IS_IN (libc) && defined SHARED +# define __memcpy_chk __redirect_memcpy_chk +# include <string.h> +# undef __memcpy_chk + +# define SYMBOL_NAME memcpy_chk +# include "ifunc-memmove.h" + +libc_ifunc_redirected (__redirect_memcpy_chk, __memcpy_chk, + IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/memmove-ia32.S b/sysdeps/i386/i686/multiarch/memmove-ia32.S new file mode 100644 index 0000000000..7f212dca66 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memmove-ia32.S @@ -0,0 +1,31 @@ +/* memmove optimized for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined SHARED && IS_IN (libc) +# define memmove __memmove_ia32 +# define __memmove_chk __memmove_chk_ia32 + +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_memmove; __GI_memmove = memmove +#endif + +#include <sysdeps/i386/i686/memmove.S> diff --git a/sysdeps/i386/i686/multiarch/memmove.S b/sysdeps/i386/i686/multiarch/memmove.S deleted file mode 100644 index 6eb418ca7f..0000000000 --- a/sysdeps/i386/i686/multiarch/memmove.S +++ /dev/null @@ -1,89 +0,0 @@ -/* Multiple versions of memmove - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in lib. */ -#if IS_IN (libc) - .text -ENTRY(memmove) - .type memmove, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__memmove_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__memmove_sse2_unaligned) - HAS_ARCH_FEATURE (Fast_Unaligned_Load) - jnz 2f - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__memmove_ssse3) - HAS_ARCH_FEATURE (Fast_Rep_String) - jz 2f - LOAD_FUNC_GOT_EAX (__memmove_ssse3_rep) -2: ret -END(memmove) - -# ifdef SHARED -# undef ENTRY -# define ENTRY(name) \ - .type __memmove_ia32, @function; \ - .p2align 4; \ - .globl __memmove_ia32; \ - .hidden __memmove_ia32; \ - __memmove_ia32: cfi_startproc; \ - CALL_MCOUNT -# else -# undef ENTRY -# define ENTRY(name) \ - .type __memmove_ia32, @function; \ - .globl __memmove_ia32; \ - .p2align 4; \ - __memmove_ia32: cfi_startproc; \ - CALL_MCOUNT -# endif - -# undef END -# define END(name) \ - cfi_endproc; .size __memmove_ia32, .-__memmove_ia32 - -# undef ENTRY_CHK -# define ENTRY_CHK(name) \ - .type __memmove_chk_ia32, @function; \ - .globl __memmove_chk_ia32; \ - .p2align 4; \ - __memmove_chk_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END_CHK -# define END_CHK(name) \ - cfi_endproc; .size __memmove_chk_ia32, .-__memmove_chk_ia32 - -# ifdef SHARED -# undef libc_hidden_builtin_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI_memmove; __GI_memmove = __memmove_ia32 -# endif -#endif - -#include "../memmove.S" diff --git a/sysdeps/i386/i686/multiarch/memmove.c b/sysdeps/i386/i686/multiarch/memmove.c new file mode 100644 index 0000000000..1cbd87f0c7 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memmove.c @@ -0,0 +1,32 @@ +/* Multiple versions of memmove. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in lib and for + DSO. In static binaries we need memmove before the initialization + happened. */ +#if defined SHARED && IS_IN (libc) +# define memmove __redirect_memmove +# include <string.h> +# undef memmove + +# define SYMBOL_NAME memmove +# include "ifunc-memmove.h" + +libc_ifunc_redirected (__redirect_memmove, memmove, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/memmove_chk-nonshared.S b/sysdeps/i386/i686/multiarch/memmove_chk-nonshared.S new file mode 100644 index 0000000000..514393d93b --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memmove_chk-nonshared.S @@ -0,0 +1,21 @@ +/* Non-shared version of memmmove_chk for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) && !defined SHARED +# include <sysdeps/i386/memmove_chk.S> +#endif diff --git a/sysdeps/i386/i686/multiarch/memmove_chk.S b/sysdeps/i386/i686/multiarch/memmove_chk.S deleted file mode 100644 index 314834c4c6..0000000000 --- a/sysdeps/i386/i686/multiarch/memmove_chk.S +++ /dev/null @@ -1,94 +0,0 @@ -/* Multiple versions of __memmove_chk - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in lib. */ -#if IS_IN (libc) - .text -ENTRY(__memmove_chk) - .type __memmove_chk, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__memmove_chk_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__memmove_chk_sse2_unaligned) - HAS_ARCH_FEATURE (Fast_Unaligned_Load) - jnz 2f - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__memmove_chk_ssse3) - HAS_ARCH_FEATURE (Fast_Rep_String) - jz 2f - LOAD_FUNC_GOT_EAX (__memmove_chk_ssse3_rep) -2: ret -END(__memmove_chk) - -# ifndef SHARED - .type __memmove_chk_sse2_unaligned, @function - .p2align 4; -__memmove_chk_sse2_unaligned: - cfi_startproc - CALL_MCOUNT - movl 12(%esp), %eax - cmpl %eax, 16(%esp) - jb __chk_fail - jmp __memmove_sse2_unaligned - cfi_endproc - .size __memmove_chk_sse2_unaligned, .-__memmove_chk_sse2_unaligned - - .type __memmove_chk_ssse3, @function - .p2align 4; -__memmove_chk_ssse3: - cfi_startproc - CALL_MCOUNT - movl 12(%esp), %eax - cmpl %eax, 16(%esp) - jb __chk_fail - jmp __memmove_ssse3 - cfi_endproc - .size __memmove_chk_ssse3, .-__memmove_chk_ssse3 - - .type __memmove_chk_ssse3_rep, @function - .p2align 4; -__memmove_chk_ssse3_rep: - cfi_startproc - CALL_MCOUNT - movl 12(%esp), %eax - cmpl %eax, 16(%esp) - jb __chk_fail - jmp __memmove_ssse3_rep - cfi_endproc - .size __memmove_chk_ssse3_rep, .-__memmove_chk_ssse3_rep - - .type __memmove_chk_ia32, @function - .p2align 4; -__memmove_chk_ia32: - cfi_startproc - CALL_MCOUNT - movl 12(%esp), %eax - cmpl %eax, 16(%esp) - jb __chk_fail - jmp __memmove_ia32 - cfi_endproc - .size __memmove_chk_ia32, .-__memmove_chk_ia32 -# endif -#endif diff --git a/sysdeps/i386/i686/multiarch/memmove_chk.c b/sysdeps/i386/i686/multiarch/memmove_chk.c new file mode 100644 index 0000000000..44339d64ec --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memmove_chk.c @@ -0,0 +1,31 @@ +/* Multiple versions of __memmove_chk + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc.so. */ +#if IS_IN (libc) && defined SHARED +# define __memmove_chk __redirect_memmove_chk +# include <string.h> +# undef __memmove_chk + +# define SYMBOL_NAME memmove_chk +# include "ifunc-memmove.h" + +libc_ifunc_redirected (__redirect_memmove_chk, __memmove_chk, + IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/strchr.S b/sysdeps/i386/i686/multiarch/mempcpy-ia32.S index 5b97b1c767..b6d5721886 100644 --- a/sysdeps/i386/i686/multiarch/strchr.S +++ b/sysdeps/i386/i686/multiarch/mempcpy-ia32.S @@ -1,7 +1,5 @@ -/* Multiple versions of strchr - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. +/* mempcpy optimized for i686. + Copyright (C) 2017 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -18,40 +16,22 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <sysdep.h> -#include <init-arch.h> +#if defined SHARED && IS_IN (libc) +# define __mempcpy __mempcpy_ia32 +# define __mempcpy_chk __mempcpy_chk_ia32 -#if IS_IN (libc) - .text -ENTRY(strchr) - .type strchr, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__strchr_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__strchr_sse2_bsf) - HAS_ARCH_FEATURE (Slow_BSF) - jz 2f - LOAD_FUNC_GOT_EAX (__strchr_sse2) -2: ret -END(strchr) - -# undef ENTRY -# define ENTRY(name) \ - .type __strchr_ia32, @function; \ - .globl __strchr_ia32; \ - .p2align 4; \ - __strchr_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __strchr_ia32, .-__strchr_ia32 +# undef libc_hidden_def # undef libc_hidden_builtin_def /* IFUNC doesn't work with the hidden functions in shared library since they will be called without setting up EBX needed for PLT which is used by IFUNC. */ +# define libc_hidden_def(name) \ + .globl __GI___mempcpy; __GI___mempcpy = __mempcpy # define libc_hidden_builtin_def(name) \ - .globl __GI_strchr; __GI_strchr = __strchr_ia32 + .globl __GI_mempcpy; __GI_mempcpy = __mempcpy + +# undef weak_alias +# define weak_alias(original, alias) #endif -#include "../../i586/strchr.S" +#include <sysdeps/i386/i686/mempcpy.S> diff --git a/sysdeps/i386/i686/multiarch/mempcpy.S b/sysdeps/i386/i686/multiarch/mempcpy.S deleted file mode 100644 index 06e377fbc9..0000000000 --- a/sysdeps/i386/i686/multiarch/mempcpy.S +++ /dev/null @@ -1,81 +0,0 @@ -/* Multiple versions of mempcpy - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in lib and for - DSO. In static binaries we need mempcpy before the initialization - happened. */ -#if defined SHARED && IS_IN (libc) - .text -ENTRY(__mempcpy) - .type __mempcpy, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__mempcpy_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__mempcpy_sse2_unaligned) - HAS_ARCH_FEATURE (Fast_Unaligned_Load) - jnz 2f - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__mempcpy_ssse3) - HAS_ARCH_FEATURE (Fast_Rep_String) - jz 2f - LOAD_FUNC_GOT_EAX (__mempcpy_ssse3_rep) -2: ret -END(__mempcpy) - -# undef ENTRY -# define ENTRY(name) \ - .type __mempcpy_ia32, @function; \ - .p2align 4; \ - .globl __mempcpy_ia32; \ - .hidden __mempcpy_ia32; \ - __mempcpy_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __mempcpy_ia32, .-__mempcpy_ia32 - -# undef ENTRY_CHK -# define ENTRY_CHK(name) \ - .type __mempcpy_chk_ia32, @function; \ - .globl __mempcpy_chk_ia32; \ - .p2align 4; \ - __mempcpy_chk_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END_CHK -# define END_CHK(name) \ - cfi_endproc; .size __mempcpy_chk_ia32, .-__mempcpy_chk_ia32 - -# undef libc_hidden_def -# undef libc_hidden_builtin_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_def(name) \ - .globl __GI_mempcpy; __GI_mempcpy = __mempcpy_ia32 -# define libc_hidden_builtin_def(name) \ - .globl __GI___mempcpy; __GI___mempcpy = __mempcpy_ia32 -#endif - -#include "../mempcpy.S" diff --git a/sysdeps/i386/i686/multiarch/wcscmp.S b/sysdeps/i386/i686/multiarch/mempcpy.c index 7118bdd4db..46d7f135ae 100644 --- a/sysdeps/i386/i686/multiarch/wcscmp.S +++ b/sysdeps/i386/i686/multiarch/mempcpy.c @@ -1,7 +1,6 @@ -/* Multiple versions of wcscmp +/* Multiple versions of mempcpy. All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. + Copyright (C) 2017 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -18,22 +17,22 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in libc and for the - DSO. In static binaries, we need wcscmp before the initialization +/* Define multiple versions only for the definition in lib and for + DSO. In static binaries we need memcpy before the initialization happened. */ -#if IS_IN (libc) - .text -ENTRY(__wcscmp) - .type __wcscmp, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__wcscmp_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__wcscmp_sse2) -2: ret -END(__wcscmp) -weak_alias (__wcscmp, wcscmp) +#if defined SHARED && IS_IN (libc) +# define mempcpy __redirect_mempcpy +# define __mempcpy __redirect___mempcpy +# define NO_MEMPCPY_STPCPY_REDIRECT +# define __NO_STRING_INLINES +# include <string.h> +# undef mempcpy +# undef __mempcpy + +# define SYMBOL_NAME mempcpy +# include "ifunc-memmove.h" + +libc_ifunc_redirected (__redirect_mempcpy, __mempcpy, IFUNC_SELECTOR ()); + +weak_alias (__mempcpy, mempcpy) #endif diff --git a/sysdeps/i386/i686/multiarch/mempcpy_chk-nonshared.S b/sysdeps/i386/i686/multiarch/mempcpy_chk-nonshared.S new file mode 100644 index 0000000000..6e31a5d5f8 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/mempcpy_chk-nonshared.S @@ -0,0 +1,21 @@ +/* Non-shared version of mempcpy_chk for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) && !defined SHARED +# include <sysdeps/i386/mempcpy_chk.S> +#endif diff --git a/sysdeps/i386/i686/multiarch/mempcpy_chk.S b/sysdeps/i386/i686/multiarch/mempcpy_chk.S deleted file mode 100644 index e13e5248a5..0000000000 --- a/sysdeps/i386/i686/multiarch/mempcpy_chk.S +++ /dev/null @@ -1,50 +0,0 @@ -/* Multiple versions of __mempcpy_chk - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in lib and for - DSO. There are no multiarch mempcpy functions for static binaries. - */ -#if IS_IN (libc) -# ifdef SHARED - .text -ENTRY(__mempcpy_chk) - .type __mempcpy_chk, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__mempcpy_chk_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__mempcpy_chk_sse2_unaligned) - HAS_ARCH_FEATURE (Fast_Unaligned_Load) - jnz 2f - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__mempcpy_chk_ssse3) - HAS_ARCH_FEATURE (Fast_Rep_String) - jz 2f - LOAD_FUNC_GOT_EAX (__mempcpy_chk_ssse3_rep) -2: ret -END(__mempcpy_chk) -# else -# include "../mempcpy_chk.S" -# endif -#endif diff --git a/sysdeps/i386/i686/multiarch/mempcpy_chk.c b/sysdeps/i386/i686/multiarch/mempcpy_chk.c new file mode 100644 index 0000000000..c73273ece3 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/mempcpy_chk.c @@ -0,0 +1,31 @@ +/* Multiple versions of __mempcpy_chk + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc.so. */ +#if IS_IN (libc) && defined SHARED +# define __mempcpy_chk __redirect_mempcpy_chk +# include <string.h> +# undef __mempcpy_chk + +# define SYMBOL_NAME mempcpy_chk +# include "ifunc-memmove.h" + +libc_ifunc_redirected (__redirect_mempcpy_chk, __mempcpy_chk, + IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/memrchr.c b/sysdeps/i386/i686/multiarch/memrchr.c new file mode 100644 index 0000000000..e8edba0d42 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memrchr.c @@ -0,0 +1,32 @@ +/* Multiple versions of memrchr. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define memrchr __redirect_memrchr +# include <string.h> +# undef memrchr + +# define SYMBOL_NAME memrchr +# include "ifunc-sse2-bsf.h" + +libc_ifunc_redirected (__redirect_memrchr, __memrchr, IFUNC_SELECTOR ()); + +weak_alias (__memrchr, memrchr) +#endif diff --git a/sysdeps/i386/i686/multiarch/memset-ia32.S b/sysdeps/i386/i686/multiarch/memset-ia32.S new file mode 100644 index 0000000000..a91f1b859c --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memset-ia32.S @@ -0,0 +1,33 @@ +/* memset optimized for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define memset __memset_ia32 +# define __memset_chk __memset_chk_ia32 + +# ifdef SHARED +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_memset; __GI_memset = memset +# endif +#endif + +#include <sysdeps/i386/i686/memset.S> diff --git a/sysdeps/i386/i686/multiarch/memset.S b/sysdeps/i386/i686/multiarch/memset.S deleted file mode 100644 index f601663a9f..0000000000 --- a/sysdeps/i386/i686/multiarch/memset.S +++ /dev/null @@ -1,75 +0,0 @@ -/* Multiple versions of memset - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in lib. */ -#if IS_IN (libc) - .text -ENTRY(memset) - .type memset, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__memset_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__memset_sse2) - HAS_ARCH_FEATURE (Fast_Rep_String) - jz 2f - LOAD_FUNC_GOT_EAX (__memset_sse2_rep) -2: ret -END(memset) - -# undef ENTRY -# define ENTRY(name) \ - .type __memset_ia32, @function; \ - .globl __memset_ia32; \ - .p2align 4; \ - __memset_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __memset_ia32, .-__memset_ia32 - -# undef ENTRY_CHK -# define ENTRY_CHK(name) \ - .type __memset_chk_ia32, @function; \ - .globl __memset_chk_ia32; \ - .p2align 4; \ - __memset_chk_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END_CHK -# define END_CHK(name) \ - cfi_endproc; .size __memset_chk_ia32, .-__memset_chk_ia32 - -# ifdef SHARED -# undef libc_hidden_builtin_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI_memset; __GI_memset = __memset_ia32 -# endif - -# undef strong_alias -# define strong_alias(original, alias) -#endif - -#include "../memset.S" diff --git a/sysdeps/i386/i686/multiarch/memset.c b/sysdeps/i386/i686/multiarch/memset.c new file mode 100644 index 0000000000..0bc0418ef6 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memset.c @@ -0,0 +1,30 @@ +/* Multiple versions of memset. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define memset __redirect_memset +# include <string.h> +# undef memset + +# define SYMBOL_NAME memset +# include "ifunc-memset.h" + +libc_ifunc_redirected (__redirect_memset, memset, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/memset_chk-nonshared.S b/sysdeps/i386/i686/multiarch/memset_chk-nonshared.S new file mode 100644 index 0000000000..038cff6373 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memset_chk-nonshared.S @@ -0,0 +1,21 @@ +/* Non-shared version of memset_chk for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) && !defined SHARED +# include <sysdeps/i386/memset_chk.S> +#endif diff --git a/sysdeps/i386/i686/multiarch/memset_chk.S b/sysdeps/i386/i686/multiarch/memset_chk.S deleted file mode 100644 index 0580991ce4..0000000000 --- a/sysdeps/i386/i686/multiarch/memset_chk.S +++ /dev/null @@ -1,78 +0,0 @@ -/* Multiple versions of __memset_chk - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in lib. */ -#if IS_IN (libc) - .text -ENTRY(__memset_chk) - .type __memset_chk, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__memset_chk_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__memset_chk_sse2) - HAS_ARCH_FEATURE (Fast_Rep_String) - jz 2f - LOAD_FUNC_GOT_EAX (__memset_chk_sse2_rep) -2: ret -END(__memset_chk) - -# ifndef SHARED - .text - .type __memset_chk_sse2, @function - .p2align 4; -__memset_chk_sse2: - cfi_startproc - CALL_MCOUNT - movl 12(%esp), %eax - cmpl %eax, 16(%esp) - jb __chk_fail - jmp __memset_sse2 - cfi_endproc - .size __memset_chk_sse2, .-__memset_chk_sse2 - - .type __memset_chk_sse2_rep, @function - .p2align 4; -__memset_chk_sse2_rep: - cfi_startproc - CALL_MCOUNT - movl 12(%esp), %eax - cmpl %eax, 16(%esp) - jb __chk_fail - jmp __memset_sse2_rep - cfi_endproc - .size __memset_chk_sse2_rep, .-__memset_chk_sse2_rep - - .type __memset_chk_ia32, @function - .p2align 4; -__memset_chk_ia32: - cfi_startproc - CALL_MCOUNT - movl 12(%esp), %eax - cmpl %eax, 16(%esp) - jb __chk_fail - jmp __memset_ia32 - cfi_endproc - .size __memset_chk_ia32, .-__memset_chk_ia32 -# endif -#endif diff --git a/sysdeps/i386/i686/multiarch/memset_chk.c b/sysdeps/i386/i686/multiarch/memset_chk.c new file mode 100644 index 0000000000..69963348c2 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/memset_chk.c @@ -0,0 +1,32 @@ +/* Multiple versions of __memset_chk + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc.so. */ +#if IS_IN (libc) && defined SHARED +# define __memset_chk __redirect_memset_chk +# include <string.h> +# undef __memset_chk + +# define SYMBOL_NAME memset_chk +# include "ifunc-memset.h" + +libc_ifunc_redirected (__redirect_memset_chk, __memset_chk, + IFUNC_SELECTOR ()); + +#endif diff --git a/sysdeps/i386/i686/multiarch/rawmemchr-ia32.S b/sysdeps/i386/i686/multiarch/rawmemchr-ia32.S new file mode 100644 index 0000000000..c18245026a --- /dev/null +++ b/sysdeps/i386/i686/multiarch/rawmemchr-ia32.S @@ -0,0 +1,35 @@ +/* rawmemchr optimized for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define __rawmemchr __rawmemchr_ia32 + +# ifdef SHARED +# undef libc_hidden_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_def(name) \ + .globl __GI___rawmemchr; __GI___rawmemchr = __rawmemchr +# endif + +# undef weak_alias +# define weak_alias(original, alias) +#endif + +#include <sysdeps/i386/rawmemchr.S> diff --git a/sysdeps/i386/i686/multiarch/rawmemchr.S b/sysdeps/i386/i686/multiarch/rawmemchr.S deleted file mode 100644 index 0a41d63ee8..0000000000 --- a/sysdeps/i386/i686/multiarch/rawmemchr.S +++ /dev/null @@ -1,65 +0,0 @@ -/* Multiple versions of rawmemchr - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -#if IS_IN (libc) - .text -ENTRY(__rawmemchr) - .type __rawmemchr, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - HAS_CPU_FEATURE (SSE2) - jz 2f - HAS_ARCH_FEATURE (Slow_BSF) - jz 3f - - LOAD_FUNC_GOT_EAX (__rawmemchr_sse2) - ret - -2: LOAD_FUNC_GOT_EAX (__rawmemchr_ia32) - ret - -3: LOAD_FUNC_GOT_EAX (__rawmemchr_sse2_bsf) - ret -END(__rawmemchr) - -weak_alias(__rawmemchr, rawmemchr) - -# undef ENTRY -# define ENTRY(name) \ - .type __rawmemchr_ia32, @function; \ - .globl __rawmemchr_ia32; \ - .p2align 4; \ - __rawmemchr_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __rawmemchr_ia32, .-__rawmemchr_ia32 - -# undef libc_hidden_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_def(name) \ - .globl __GI___rawmemchr; __GI___rawmemchr = __rawmemchr_ia32 - -#endif -#include "../../rawmemchr.S" diff --git a/sysdeps/i386/i686/multiarch/rawmemchr.c b/sysdeps/i386/i686/multiarch/rawmemchr.c new file mode 100644 index 0000000000..1cbc4531e8 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/rawmemchr.c @@ -0,0 +1,35 @@ +/* Multiple versions of rawmemchr. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define rawmemchr __redirect_rawmemchr +# define __rawmemchr __redirect___rawmemchr +# include <string.h> +# undef rawmemchr +# undef __rawmemchr + +# define SYMBOL_NAME rawmemchr +# include "ifunc-sse2-bsf.h" + +libc_ifunc_redirected (__redirect_rawmemchr, __rawmemchr, + IFUNC_SELECTOR ()); + +weak_alias (__rawmemchr, rawmemchr) +#endif diff --git a/sysdeps/i386/i686/multiarch/stpcpy-ia32.S b/sysdeps/i386/i686/multiarch/stpcpy-ia32.S new file mode 100644 index 0000000000..679d530690 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/stpcpy-ia32.S @@ -0,0 +1,39 @@ +/* stpcpy optimized for i686. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define __stpcpy __stpcpy_ia32 + +# ifdef SHARED +# undef libc_hidden_def +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_def(name) \ + .globl __GI___stpcpy; __GI___stpcpy = __stpcpy +# define libc_hidden_builtin_def(name) \ + .globl __GI_stpcpy; __GI_stpcpy = __stpcpy +# endif + +# undef weak_alias +# define weak_alias(original, alias) +#endif + +#include <sysdeps/i386/i586/stpcpy.S> diff --git a/sysdeps/i386/i686/multiarch/stpcpy.S b/sysdeps/i386/i686/multiarch/stpcpy.S deleted file mode 100644 index ee81ab6ae3..0000000000 --- a/sysdeps/i386/i686/multiarch/stpcpy.S +++ /dev/null @@ -1,9 +0,0 @@ -/* Multiple versions of stpcpy - All versions must be listed in ifunc-impl-list.c. */ -#define USE_AS_STPCPY -#define STRCPY __stpcpy -#include "strcpy.S" - -weak_alias (__stpcpy, stpcpy) -libc_hidden_def (__stpcpy) -libc_hidden_builtin_def (stpcpy) diff --git a/sysdeps/i386/i686/multiarch/stpcpy.c b/sysdeps/i386/i686/multiarch/stpcpy.c new file mode 100644 index 0000000000..32553b9ed2 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/stpcpy.c @@ -0,0 +1,36 @@ +/* Multiple versions of stpcpy. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define stpcpy __redirect_stpcpy +# define __stpcpy __redirect___stpcpy +# define NO_MEMPCPY_STPCPY_REDIRECT +# define __NO_STRING_INLINES +# include <string.h> +# undef stpcpy +# undef __stpcpy + +# define SYMBOL_NAME stpcpy +# include "ifunc-sse2-ssse3.h" + +libc_ifunc_redirected (__redirect_stpcpy, __stpcpy, IFUNC_SELECTOR ()); + +weak_alias (__stpcpy, stpcpy) +#endif diff --git a/sysdeps/i386/i686/multiarch/memrchr.S b/sysdeps/i386/i686/multiarch/stpncpy-ia32.S index d4253a553b..d1be1fcf96 100644 --- a/sysdeps/i386/i686/multiarch/memrchr.S +++ b/sysdeps/i386/i686/multiarch/stpncpy-ia32.S @@ -1,7 +1,6 @@ -/* Multiple versions of memrchr +/* stpncpy optimized for i686. All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. + Copyright (C) 2017 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -18,28 +17,21 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <sysdep.h> -#include <init-arch.h> - #if IS_IN (libc) - .text -ENTRY(__memrchr) - .type __memrchr, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - HAS_CPU_FEATURE (SSE2) - jz 2f - HAS_ARCH_FEATURE (Slow_BSF) - jz 3f - - LOAD_FUNC_GOT_EAX (__memrchr_sse2) - ret - -2: LOAD_FUNC_GOT_EAX (__memrchr_ia32) - ret - -3: LOAD_FUNC_GOT_EAX (__memrchr_sse2_bsf) - ret -END(__memrchr) - -weak_alias(__memrchr, memrchr) +# define __stpncpy __stpncpy_ia32 + +# ifdef SHARED +# undef libc_hidden_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_def(name) \ + .globl __GI___stpncpy; __GI___stpncpy = __stpncpy + +# undef weak_alias +# define weak_alias(original, alias) +# endif + #endif + +#include <sysdeps/i386/stpncpy.S> diff --git a/sysdeps/i386/i686/multiarch/stpncpy.S b/sysdeps/i386/i686/multiarch/stpncpy.S deleted file mode 100644 index 2698ca6a8c..0000000000 --- a/sysdeps/i386/i686/multiarch/stpncpy.S +++ /dev/null @@ -1,8 +0,0 @@ -/* Multiple versions of stpncpy - All versions must be listed in ifunc-impl-list.c. */ -#define STRCPY __stpncpy -#define USE_AS_STPCPY -#define USE_AS_STRNCPY -#include "strcpy.S" - -weak_alias (__stpncpy, stpncpy) diff --git a/sysdeps/i386/i686/multiarch/stpncpy.c b/sysdeps/i386/i686/multiarch/stpncpy.c new file mode 100644 index 0000000000..ec9b888f32 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/stpncpy.c @@ -0,0 +1,34 @@ +/* Multiple versions of stpncpy. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define stpncpy __redirect_stpncpy +# define __stpncpy __redirect___stpncpy +# include <string.h> +# undef stpncpy +# undef __stpncpy + +# define SYMBOL_NAME stpncpy +# include "ifunc-sse2-ssse3.h" + +libc_ifunc_redirected (__redirect_stpncpy, __stpncpy, IFUNC_SELECTOR ()); + +weak_alias (__stpncpy, stpncpy) +#endif diff --git a/sysdeps/i386/i686/multiarch/strcasecmp.S b/sysdeps/i386/i686/multiarch/strcasecmp.c index ec59276408..fd9fae3d3f 100644 --- a/sysdeps/i386/i686/multiarch/strcasecmp.S +++ b/sysdeps/i386/i686/multiarch/strcasecmp.c @@ -1,6 +1,6 @@ -/* Entry point for multi-version x86 strcasecmp. +/* Multiple versions of strcasecmp. All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. + Copyright (C) 2017 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -17,23 +17,19 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <sysdep.h> -#include <init-arch.h> +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strcasecmp __redirect_strcasecmp +# define __strcasecmp __redirect___strcasecmp +# include <string.h> +# undef strcasecmp +# undef __strcasecmp - .text -ENTRY(__strcasecmp) - .type __strcasecmp, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__strcasecmp_ia32) - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__strcasecmp_ssse3) - HAS_CPU_FEATURE (SSE4_2) - jz 2f - HAS_ARCH_FEATURE (Slow_SSE4_2) - jnz 2f - LOAD_FUNC_GOT_EAX (__strcasecmp_sse4_2) -2: ret -END(__strcasecmp) +# define SYMBOL_NAME strcasecmp +# include "ifunc-ssse3-sse4_2.h" + +libc_ifunc_redirected (__redirect_strcasecmp, __strcasecmp, + IFUNC_SELECTOR ()); weak_alias (__strcasecmp, strcasecmp) +#endif diff --git a/sysdeps/i386/i686/multiarch/strcasecmp_l.S b/sysdeps/i386/i686/multiarch/strcasecmp_l.S deleted file mode 100644 index 711c09b0dc..0000000000 --- a/sysdeps/i386/i686/multiarch/strcasecmp_l.S +++ /dev/null @@ -1,7 +0,0 @@ -/* Multiple versions of strcasecmp_l - All versions must be listed in ifunc-impl-list.c. */ -#define STRCMP __strcasecmp_l -#define USE_AS_STRCASECMP_L -#include "strcmp.S" - -weak_alias (__strcasecmp_l, strcasecmp_l) diff --git a/sysdeps/i386/i686/multiarch/wmemcmp.S b/sysdeps/i386/i686/multiarch/strcasecmp_l.c index 1b9a54a413..56f21a7535 100644 --- a/sysdeps/i386/i686/multiarch/wmemcmp.S +++ b/sysdeps/i386/i686/multiarch/strcasecmp_l.c @@ -1,7 +1,6 @@ -/* Multiple versions of wmemcmp +/* Multiple versions of strcasecmp_l. All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. + Copyright (C) 2017 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -18,23 +17,19 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <sysdep.h> -#include <init-arch.h> +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strcasecmp_l __redirect_strcasecmp_l +# define __strcasecmp_l __redirect___strcasecmp_l +# include <string.h> +# undef strcasecmp_l +# undef __strcasecmp_l -/* Define multiple versions only for the definition in libc. */ +# define SYMBOL_NAME strcasecmp_l +# include "ifunc-ssse3-sse4_2.h" -#if IS_IN (libc) - .text -ENTRY(wmemcmp) - .type wmemcmp, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__wmemcmp_ia32) - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__wmemcmp_ssse3) - HAS_CPU_FEATURE (SSE4_2) - jz 2f - LOAD_FUNC_GOT_EAX (__wmemcmp_sse4_2) -2: ret -END(wmemcmp) +libc_ifunc_redirected (__redirect_strcasecmp_l, __strcasecmp_l, + IFUNC_SELECTOR ()); + +weak_alias (__strcasecmp_l, strcasecmp_l) #endif diff --git a/sysdeps/i386/i686/multiarch/strcat-ia32.S b/sysdeps/i386/i686/multiarch/strcat-ia32.S new file mode 100644 index 0000000000..81d0c8dcf9 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strcat-ia32.S @@ -0,0 +1,33 @@ +/* strcat optimized for i686. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define strcat __strcat_ia32 + +# ifdef SHARED +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_strcat; __GI_strcat = strcat +# endif +#endif + +#include <sysdeps/i386/strcat.S> diff --git a/sysdeps/i386/i686/multiarch/strcat.S b/sysdeps/i386/i686/multiarch/strcat.S deleted file mode 100644 index 8412cb6f23..0000000000 --- a/sysdeps/i386/i686/multiarch/strcat.S +++ /dev/null @@ -1,92 +0,0 @@ -/* Multiple versions of strcat - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -#ifndef USE_AS_STRNCAT -# ifndef STRCAT -# define STRCAT strcat -# endif -#endif - -#ifdef USE_AS_STRNCAT -# define STRCAT_SSSE3 __strncat_ssse3 -# define STRCAT_SSE2 __strncat_sse2 -# define STRCAT_IA32 __strncat_ia32 -# define __GI_STRCAT __GI_strncat -#else -# define STRCAT_SSSE3 __strcat_ssse3 -# define STRCAT_SSE2 __strcat_sse2 -# define STRCAT_IA32 __strcat_ia32 -# define __GI_STRCAT __GI_strcat -#endif - - -/* Define multiple versions only for the definition in libc. Don't - define multiple versions for strncat in static library since we - need strncat before the initialization happened. */ -#if IS_IN (libc) - - .text -ENTRY(STRCAT) - .type STRCAT, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (STRCAT_IA32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (STRCAT_SSE2) - HAS_ARCH_FEATURE (Fast_Unaligned_Load) - jnz 2f - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (STRCAT_SSSE3) -2: ret -END(STRCAT) - -# undef ENTRY -# define ENTRY(name) \ - .type STRCAT_IA32, @function; \ - .align 16; \ - .globl STRCAT_IA32; \ - .hidden STRCAT_IA32; \ - STRCAT_IA32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size STRCAT_IA32, .-STRCAT_IA32 - -# ifdef SHARED -# undef libc_hidden_builtin_def -/* It doesn't make sense to send libc-internal strcat calls through a PLT. - The speedup we get from using SSSE3 instruction is likely eaten away - by the indirect call in the PLT. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI_STRCAT; __GI_STRCAT = STRCAT_IA32 -# undef libc_hidden_def -# define libc_hidden_def(name) \ - .globl __GI___STRCAT; __GI___STRCAT = STRCAT_IA32 - -# endif -#endif - -#ifndef USE_AS_STRNCAT -# include "../../strcat.S" -#endif diff --git a/sysdeps/i386/i686/multiarch/strcat.c b/sysdeps/i386/i686/multiarch/strcat.c new file mode 100644 index 0000000000..f41cecf41b --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strcat.c @@ -0,0 +1,30 @@ +/* Multiple versions of strcat. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strcat __redirect_strcat +# include <string.h> +# undef strcat + +# define SYMBOL_NAME strcat +# include "ifunc-sse2-ssse3.h" + +libc_ifunc_redirected (__redirect_strcat, strcat, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/strchr-ia32.S b/sysdeps/i386/i686/multiarch/strchr-ia32.S new file mode 100644 index 0000000000..4e55bf9812 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strchr-ia32.S @@ -0,0 +1,35 @@ +/* strchr optimized for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define strchr __strchr_ia32 + +# ifdef SHARED +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_strchr; __GI_strchr = strchr +# endif + +# undef weak_alias +# define weak_alias(original, alias) +#endif + +#include <sysdeps/i386/i586/strchr.S> diff --git a/sysdeps/i386/i686/multiarch/strchr.c b/sysdeps/i386/i686/multiarch/strchr.c new file mode 100644 index 0000000000..2029250797 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strchr.c @@ -0,0 +1,32 @@ +/* Multiple versions of strchr. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strchr __redirect_strchr +# include <string.h> +# undef strchr + +# define SYMBOL_NAME strchr +# include "ifunc-sse2-bsf.h" + +libc_ifunc_redirected (__redirect_strchr, strchr, IFUNC_SELECTOR ()); + +weak_alias (strchr, index) +#endif diff --git a/sysdeps/i386/i686/multiarch/strcmp-ia32.S b/sysdeps/i386/i686/multiarch/strcmp-ia32.S new file mode 100644 index 0000000000..2e87f19986 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strcmp-ia32.S @@ -0,0 +1,33 @@ +/* strcmp optimized for i686. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define strcmp __strcmp_ia32 + +# ifdef SHARED +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_strcmp; __GI_strcmp = strcmp +# endif +#endif + +#include <sysdeps/i386/i686/strcmp.S> diff --git a/sysdeps/i386/i686/multiarch/strcmp.S b/sysdeps/i386/i686/multiarch/strcmp.S deleted file mode 100644 index 56de25a4b7..0000000000 --- a/sysdeps/i386/i686/multiarch/strcmp.S +++ /dev/null @@ -1,95 +0,0 @@ -/* Multiple versions of strcmp - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -#ifdef USE_AS_STRNCMP -# define STRCMP strncmp -# define __GI_STRCMP __GI_strncmp -# define __STRCMP_IA32 __strncmp_ia32 -# define __STRCMP_SSSE3 __strncmp_ssse3 -# define __STRCMP_SSE4_2 __strncmp_sse4_2 -#elif defined USE_AS_STRCASECMP_L -# define STRCMP __strcasecmp_l -# define __GI_STRCMP __GI_strcasecmp_l -# define __STRCMP_IA32 __strcasecmp_l_ia32 -# define __STRCMP_SSSE3 __strcasecmp_l_ssse3 -# define __STRCMP_SSE4_2 __strcasecmp_l_sse4_2 -#elif defined USE_AS_STRNCASECMP_L -# define STRCMP __strncasecmp_l -# define __GI_STRCMP __GI_strncasecmp_l -# define __STRCMP_IA32 __strncasecmp_l_ia32 -# define __STRCMP_SSSE3 __strncasecmp_l_ssse3 -# define __STRCMP_SSE4_2 __strncasecmp_l_sse4_2 -#else -# define STRCMP strcmp -# define __GI_STRCMP __GI_strcmp -# define __STRCMP_IA32 __strcmp_ia32 -# define __STRCMP_SSSE3 __strcmp_ssse3 -# define __STRCMP_SSE4_2 __strcmp_sse4_2 -#endif - -/* Define multiple versions only for the definition in libc. Don't - define multiple versions for strncmp in static library since we - need strncmp before the initialization happened. */ -#if (defined SHARED || !defined USE_AS_STRNCMP) && IS_IN (libc) - .text -ENTRY(STRCMP) - .type STRCMP, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__STRCMP_IA32) - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__STRCMP_SSSE3) - HAS_CPU_FEATURE (SSE4_2) - jz 2f - HAS_ARCH_FEATURE (Slow_SSE4_2) - jnz 2f - LOAD_FUNC_GOT_EAX (__STRCMP_SSE4_2) -2: ret -END(STRCMP) - -# undef ENTRY -# define ENTRY(name) \ - .type __STRCMP_IA32, @function; \ - .p2align 4; \ - .globl __STRCMP_IA32; \ - .hidden __STRCMP_IA32; \ - __STRCMP_IA32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __STRCMP_IA32, .-__STRCMP_IA32 - -# ifdef SHARED -# undef libc_hidden_builtin_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI_STRCMP; __GI_STRCMP = __STRCMP_IA32 -# endif -#endif - -#if !defined USE_AS_STRNCMP && !defined USE_AS_STRCASECMP_L \ - && !defined USE_AS_STRNCASECMP_L -# include "../strcmp.S" -#endif diff --git a/sysdeps/i386/i686/multiarch/strcmp.c b/sysdeps/i386/i686/multiarch/strcmp.c new file mode 100644 index 0000000000..4388181c82 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strcmp.c @@ -0,0 +1,30 @@ +/* Multiple versions of strcmp. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strcmp __redirect_strcmp +# include <string.h> +# undef strcmp + +# define SYMBOL_NAME strcmp +# include "ifunc-ssse3-sse4_2.h" + +libc_ifunc_redirected (__redirect_strcmp, strcmp, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/strcpy-ia32.S b/sysdeps/i386/i686/multiarch/strcpy-ia32.S new file mode 100644 index 0000000000..14205b8764 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strcpy-ia32.S @@ -0,0 +1,33 @@ +/* strcpy optimized for i686. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define strcpy __strcpy_ia32 + +# ifdef SHARED +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_strcpy; __GI_strcpy = strcpy +# endif +#endif + +#include <sysdeps/i386/i586/strcpy.S> diff --git a/sysdeps/i386/i686/multiarch/strcpy.S b/sysdeps/i386/i686/multiarch/strcpy.S deleted file mode 100644 index ffbc03c6d5..0000000000 --- a/sysdeps/i386/i686/multiarch/strcpy.S +++ /dev/null @@ -1,116 +0,0 @@ -/* Multiple versions of strcpy - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -#if !defined (USE_AS_STPCPY) && !defined (USE_AS_STRNCPY) -# ifndef STRCPY -# define STRCPY strcpy -# endif -#endif - -#ifdef USE_AS_STPCPY -# ifdef USE_AS_STRNCPY -# define STRCPY_SSSE3 __stpncpy_ssse3 -# define STRCPY_SSE2 __stpncpy_sse2 -# define STRCPY_IA32 __stpncpy_ia32 -# define __GI_STRCPY __GI_stpncpy -# define __GI___STRCPY __GI___stpncpy -# else -# define STRCPY_SSSE3 __stpcpy_ssse3 -# define STRCPY_SSE2 __stpcpy_sse2 -# define STRCPY_IA32 __stpcpy_ia32 -# define __GI_STRCPY __GI_stpcpy -# define __GI___STRCPY __GI___stpcpy -# endif -#else -# ifdef USE_AS_STRNCPY -# define STRCPY_SSSE3 __strncpy_ssse3 -# define STRCPY_SSE2 __strncpy_sse2 -# define STRCPY_IA32 __strncpy_ia32 -# define __GI_STRCPY __GI_strncpy -# else -# define STRCPY_SSSE3 __strcpy_ssse3 -# define STRCPY_SSE2 __strcpy_sse2 -# define STRCPY_IA32 __strcpy_ia32 -# define __GI_STRCPY __GI_strcpy -# endif -#endif - - -/* Define multiple versions only for the definition in libc. Don't - define multiple versions for strncpy in static library since we - need strncpy before the initialization happened. */ -#if IS_IN (libc) - - .text -ENTRY(STRCPY) - .type STRCPY, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (STRCPY_IA32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (STRCPY_SSE2) - HAS_ARCH_FEATURE (Fast_Unaligned_Load) - jnz 2f - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (STRCPY_SSSE3) -2: ret -END(STRCPY) - -# undef ENTRY -# define ENTRY(name) \ - .type STRCPY_IA32, @function; \ - .align 16; \ - .globl STRCPY_IA32; \ - .hidden STRCPY_IA32; \ - STRCPY_IA32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size STRCPY_IA32, .-STRCPY_IA32 - -# ifdef SHARED -# undef libc_hidden_builtin_def -/* It doesn't make sense to send libc-internal strcpy calls through a PLT. - The speedup we get from using SSSE3 instruction is likely eaten away - by the indirect call in the PLT. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI_STRCPY; __GI_STRCPY = STRCPY_IA32 -# undef libc_hidden_def -# define libc_hidden_def(name) \ - .globl __GI___STRCPY; __GI___STRCPY = STRCPY_IA32 - -# endif -#endif - -#ifdef USE_AS_STPCPY -# ifdef USE_AS_STRNCPY -# include "../../stpncpy.S" -# else -# include "../../i586/stpcpy.S" -# endif -#else -# ifndef USE_AS_STRNCPY -# include "../../i586/strcpy.S" -# endif -#endif diff --git a/sysdeps/i386/i686/multiarch/strcpy.c b/sysdeps/i386/i686/multiarch/strcpy.c new file mode 100644 index 0000000000..e36d1740f8 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strcpy.c @@ -0,0 +1,30 @@ +/* Multiple versions of strcpy. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strcpy __redirect_strcpy +# include <string.h> +# undef strcpy + +# define SYMBOL_NAME strcpy +# include "ifunc-sse2-ssse3.h" + +libc_ifunc_redirected (__redirect_strcpy, strcpy, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/strcspn-ia32.S b/sysdeps/i386/i686/multiarch/strcspn-ia32.S new file mode 100644 index 0000000000..483a38deb0 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strcspn-ia32.S @@ -0,0 +1,33 @@ +/* strcspn optimized for i686. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define strcspn __strcspn_ia32 + +# ifdef SHARED +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_strcspn; __GI_strcspn = strcspn +# endif +#endif + +#include <sysdeps/i386/strcspn.S> diff --git a/sysdeps/i386/i686/multiarch/strcspn.S b/sysdeps/i386/i686/multiarch/strcspn.S deleted file mode 100644 index 21e5093924..0000000000 --- a/sysdeps/i386/i686/multiarch/strcspn.S +++ /dev/null @@ -1,75 +0,0 @@ -/* Multiple versions of strcspn - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2009-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <config.h> -#include <sysdep.h> -#include <init-arch.h> - -#ifdef USE_AS_STRPBRK -#define STRCSPN_SSE42 __strpbrk_sse42 -#define STRCSPN_IA32 __strpbrk_ia32 -#define __GI_STRCSPN __GI_strpbrk -#else -#ifndef STRCSPN -#define STRCSPN strcspn -#define STRCSPN_SSE42 __strcspn_sse42 -#define STRCSPN_IA32 __strcspn_ia32 -#define __GI_STRCSPN __GI_strcspn -#endif -#endif - -/* Define multiple versions only for the definition in libc. Don't - define multiple versions for strpbrk in static library since we - need strpbrk before the initialization happened. */ -#if (defined SHARED || !defined USE_AS_STRPBRK) && IS_IN (libc) - .text -ENTRY(STRCSPN) - .type STRCSPN, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (STRCSPN_IA32) - HAS_CPU_FEATURE (SSE4_2) - jz 2f - LOAD_FUNC_GOT_EAX (STRCSPN_SSE42) -2: ret -END(STRCSPN) - -# undef ENTRY -# define ENTRY(name) \ - .type STRCSPN_IA32, @function; \ - .globl STRCSPN_IA32; \ - .p2align 4; \ - STRCSPN_IA32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size STRCSPN_IA32, .-STRCSPN_IA32 -# undef libc_hidden_builtin_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI_STRCSPN; __GI_STRCSPN = STRCSPN_IA32 -#endif - -#ifdef USE_AS_STRPBRK -#include "../../strpbrk.S" -#else -#include "../../strcspn.S" -#endif diff --git a/sysdeps/i386/i686/multiarch/strcspn.c b/sysdeps/i386/i686/multiarch/strcspn.c new file mode 100644 index 0000000000..3cf1aa8e64 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strcspn.c @@ -0,0 +1,30 @@ +/* Multiple versions of strcspn. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strcspn __redirect_strcspn +# include <string.h> +# undef strcspn + +# define SYMBOL_NAME strcspn +# include "ifunc-sse4_2.h" + +libc_ifunc_redirected (__redirect_strcspn, strcspn, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/strlen-ia32.S b/sysdeps/i386/i686/multiarch/strlen-ia32.S new file mode 100644 index 0000000000..c5117833af --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strlen-ia32.S @@ -0,0 +1,30 @@ +/* strlen optimized for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if defined SHARED && IS_IN (libc) +# define strlen __strlen_ia32 + +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_strlen; __GI_strlen = strlen +#endif + +#include <sysdeps/i386/i586/strlen.S> diff --git a/sysdeps/i386/i686/multiarch/strlen.S b/sysdeps/i386/i686/multiarch/strlen.S deleted file mode 100644 index 77cf6bcdb0..0000000000 --- a/sysdeps/i386/i686/multiarch/strlen.S +++ /dev/null @@ -1,60 +0,0 @@ -/* Multiple versions of strlen - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2009-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in libc and for the - DSO. In static binaries, we need strlen before the initialization - happened. */ -#if defined SHARED && IS_IN (libc) - .text -ENTRY(strlen) - .type strlen, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__strlen_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__strlen_sse2_bsf) - HAS_ARCH_FEATURE (Slow_BSF) - jz 2f - LOAD_FUNC_GOT_EAX (__strlen_sse2) -2: ret -END(strlen) - -# undef ENTRY -# define ENTRY(name) \ - .type __strlen_ia32, @function; \ - .globl __strlen_ia32; \ - .p2align 4; \ - __strlen_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __strlen_ia32, .-__strlen_ia32 -# undef libc_hidden_builtin_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI_strlen; __GI_strlen = __strlen_ia32 -#endif - -#include "../../i586/strlen.S" diff --git a/sysdeps/i386/i686/multiarch/strlen.c b/sysdeps/i386/i686/multiarch/strlen.c new file mode 100644 index 0000000000..2968bcce83 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strlen.c @@ -0,0 +1,30 @@ +/* Multiple versions of strlen. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc.so. */ +#if defined SHARED && IS_IN (libc) +# define strlen __redirect_strlen +# include <string.h> +# undef strlen + +# define SYMBOL_NAME strlen +# include "ifunc-sse2-bsf.h" + +libc_ifunc_redirected (__redirect_strlen, strlen, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/strncase.S b/sysdeps/i386/i686/multiarch/strncase.c index a56e63a566..68922e475a 100644 --- a/sysdeps/i386/i686/multiarch/strncase.S +++ b/sysdeps/i386/i686/multiarch/strncase.c @@ -1,6 +1,6 @@ -/* Entry point for multi-version x86 strncasecmp. +/* Multiple versions of strncasecmp. All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. + Copyright (C) 2017 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -17,23 +17,19 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <sysdep.h> -#include <init-arch.h> +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strncasecmp __redirect_strncasecmp +# define __strncasecmp __redirect___strncasecmp +# include <string.h> +# undef strncasecmp +# undef __strncasecmp - .text -ENTRY(__strncasecmp) - .type __strncasecmp, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__strncasecmp_ia32) - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__strncasecmp_ssse3) - HAS_CPU_FEATURE (SSE4_2) - jz 2f - HAS_ARCH_FEATURE (Slow_SSE4_2) - jnz 2f - LOAD_FUNC_GOT_EAX (__strncasecmp_sse4_2) -2: ret -END(__strncasecmp) +# define SYMBOL_NAME strncasecmp +# include "ifunc-ssse3-sse4_2.h" + +libc_ifunc_redirected (__redirect_strncasecmp, __strncasecmp, + IFUNC_SELECTOR ()); weak_alias (__strncasecmp, strncasecmp) +#endif diff --git a/sysdeps/i386/i686/multiarch/strncase_l.S b/sysdeps/i386/i686/multiarch/strncase_l.S deleted file mode 100644 index 8a74ee8574..0000000000 --- a/sysdeps/i386/i686/multiarch/strncase_l.S +++ /dev/null @@ -1,7 +0,0 @@ -/* Multiple versions of strncasecmp_l - All versions must be listed in ifunc-impl-list.c. */ -#define STRCMP __strncasecmp_l -#define USE_AS_STRNCASECMP_L -#include "strcmp.S" - -weak_alias (__strncasecmp_l, strncasecmp_l) diff --git a/sysdeps/i386/i686/multiarch/strncase_l.c b/sysdeps/i386/i686/multiarch/strncase_l.c new file mode 100644 index 0000000000..ab25cb9050 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strncase_l.c @@ -0,0 +1,35 @@ +/* Multiple versions of strncasecmp_l. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strncasecmp_l __redirect_strncasecmp_l +# define __strncasecmp_l __redirect___strncasecmp_l +# include <string.h> +# undef strncasecmp_l +# undef __strncasecmp_l + +# define SYMBOL_NAME strncasecmp_l +# include "ifunc-ssse3-sse4_2.h" + +libc_ifunc_redirected (__redirect_strncasecmp_l, __strncasecmp_l, + IFUNC_SELECTOR ()); + +weak_alias (__strncasecmp_l, strncasecmp_l) +#endif diff --git a/sysdeps/i386/i686/multiarch/strncat.S b/sysdeps/i386/i686/multiarch/strncat.S deleted file mode 100644 index 5c1bf41453..0000000000 --- a/sysdeps/i386/i686/multiarch/strncat.S +++ /dev/null @@ -1,5 +0,0 @@ -/* Multiple versions of strncat - All versions must be listed in ifunc-impl-list.c. */ -#define STRCAT strncat -#define USE_AS_STRNCAT -#include "strcat.S" diff --git a/sysdeps/i386/i686/multiarch/strncat.c b/sysdeps/i386/i686/multiarch/strncat.c new file mode 100644 index 0000000000..5acad3d4d3 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strncat.c @@ -0,0 +1,30 @@ +/* Multiple versions of strcat. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strncat __redirect_strncat +# include <string.h> +# undef strncat + +# define SYMBOL_NAME strncat +# include "ifunc-sse2-ssse3.h" + +libc_ifunc_redirected (__redirect_strncat, strncat, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/strncmp.S b/sysdeps/i386/i686/multiarch/strncmp.S deleted file mode 100644 index 150d4786d2..0000000000 --- a/sysdeps/i386/i686/multiarch/strncmp.S +++ /dev/null @@ -1,5 +0,0 @@ -/* Multiple versions of strncmp - All versions must be listed in ifunc-impl-list.c. */ -#define USE_AS_STRNCMP -#define STRCMP strncmp -#include "strcmp.S" diff --git a/sysdeps/i386/i686/multiarch/strncmp.c b/sysdeps/i386/i686/multiarch/strncmp.c new file mode 100644 index 0000000000..3fea85f3a4 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strncmp.c @@ -0,0 +1,30 @@ +/* Multiple versions of strncmp. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc.so. */ +#if defined SHARED && IS_IN (libc) +# define strncmp __redirect_strncmp +# include <string.h> +# undef strncmp + +# define SYMBOL_NAME strncmp +# include "ifunc-ssse3-sse4_2.h" + +libc_ifunc_redirected (__redirect_strncmp, strncmp, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/strncpy.S b/sysdeps/i386/i686/multiarch/strncpy.S deleted file mode 100644 index 9c257efc6e..0000000000 --- a/sysdeps/i386/i686/multiarch/strncpy.S +++ /dev/null @@ -1,5 +0,0 @@ -/* Multiple versions of strncpy - All versions must be listed in ifunc-impl-list.c. */ -#define USE_AS_STRNCPY -#define STRCPY strncpy -#include "strcpy.S" diff --git a/sysdeps/i386/i686/multiarch/strncpy.c b/sysdeps/i386/i686/multiarch/strncpy.c new file mode 100644 index 0000000000..24404f6fa7 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strncpy.c @@ -0,0 +1,31 @@ +/* Multiple versions of strncpy. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define _HAVE_STRING_ARCH_strncpy 1 +# define strncpy __redirect_strncpy +# include <string.h> +# undef strncpy + +# define SYMBOL_NAME strncpy +# include "ifunc-sse2-ssse3.h" + +libc_ifunc_redirected (__redirect_strncpy, strncpy, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/wcscpy.S b/sysdeps/i386/i686/multiarch/strnlen.c index cfc97dd87c..83930bc086 100644 --- a/sysdeps/i386/i686/multiarch/wcscpy.S +++ b/sysdeps/i386/i686/multiarch/strnlen.c @@ -1,7 +1,6 @@ -/* Multiple versions of wcscpy +/* Multiple versions of strnlen. All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. + Copyright (C) 2017 Free Software Foundation, Inc. This file is part of the GNU C Library. The GNU C Library is free software; you can redistribute it and/or @@ -18,19 +17,18 @@ License along with the GNU C Library; if not, see <http://www.gnu.org/licenses/>. */ -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in libc. */ +/* Define multiple versions only for the definition in libc. */ #if IS_IN (libc) - .text -ENTRY(wcscpy) - .type wcscpy, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__wcscpy_ia32) - HAS_CPU_FEATURE (SSSE3) - jz 2f - LOAD_FUNC_GOT_EAX (__wcscpy_ssse3) -2: ret -END(wcscpy) +# define strnlen __redirect_strnlen +# define __strnlen __redirect___strnlen +# include <string.h> +# undef __strnlen +# undef strnlen + +# define SYMBOL_NAME strnlen +# include "ifunc-sse2.h" + +libc_ifunc_redirected (__redirect_strnlen, __strnlen, IFUNC_SELECTOR ()); + +weak_alias (__strnlen, strnlen); #endif diff --git a/sysdeps/i386/i686/multiarch/strpbrk-ia32.S b/sysdeps/i386/i686/multiarch/strpbrk-ia32.S new file mode 100644 index 0000000000..506043fe62 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strpbrk-ia32.S @@ -0,0 +1,33 @@ +/* strpbrk optimized for i686. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define strpbrk __strpbrk_ia32 + +# ifdef SHARED +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_strpbrk; __GI_strpbrk = strpbrk +# endif +#endif + +#include <sysdeps/i386/strpbrk.S> diff --git a/sysdeps/i386/i686/multiarch/strpbrk.S b/sysdeps/i386/i686/multiarch/strpbrk.S deleted file mode 100644 index 7201d6376f..0000000000 --- a/sysdeps/i386/i686/multiarch/strpbrk.S +++ /dev/null @@ -1,5 +0,0 @@ -/* Multiple versions of strpbrk - All versions must be listed in ifunc-impl-list.c. */ -#define STRCSPN strpbrk -#define USE_AS_STRPBRK -#include "strcspn.S" diff --git a/sysdeps/i386/i686/multiarch/strpbrk.c b/sysdeps/i386/i686/multiarch/strpbrk.c new file mode 100644 index 0000000000..50bbb16edb --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strpbrk.c @@ -0,0 +1,30 @@ +/* Multiple versions of strpbrk. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strpbrk __redirect_strpbrk +# include <string.h> +# undef strpbrk + +# define SYMBOL_NAME strpbrk +# include "ifunc-sse4_2.h" + +libc_ifunc_redirected (__redirect_strpbrk, strpbrk, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/strrchr-ia32.S b/sysdeps/i386/i686/multiarch/strrchr-ia32.S new file mode 100644 index 0000000000..a0effaa487 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strrchr-ia32.S @@ -0,0 +1,35 @@ +/* strrchr optimized for i686. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define strrchr __strrchr_ia32 + +# ifdef SHARED +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_strrchr; __GI_strrchr = strrchr +# endif + +# undef weak_alias +# define weak_alias(original, alias) +#endif + +#include <sysdeps/i386/strrchr.S> diff --git a/sysdeps/i386/i686/multiarch/strrchr.S b/sysdeps/i386/i686/multiarch/strrchr.S deleted file mode 100644 index d9281eaeae..0000000000 --- a/sysdeps/i386/i686/multiarch/strrchr.S +++ /dev/null @@ -1,57 +0,0 @@ -/* Multiple versions of strrchr - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2011-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <sysdep.h> -#include <init-arch.h> - -#if IS_IN (libc) - .text -ENTRY(strrchr) - .type strrchr, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__strrchr_ia32) - HAS_CPU_FEATURE (SSE2) - jz 2f - LOAD_FUNC_GOT_EAX (__strrchr_sse2_bsf) - HAS_ARCH_FEATURE (Slow_BSF) - jz 2f - LOAD_FUNC_GOT_EAX (__strrchr_sse2) -2: ret -END(strrchr) - -# undef ENTRY -# define ENTRY(name) \ - .type __strrchr_ia32, @function; \ - .globl __strrchr_ia32; \ - .p2align 4; \ - __strrchr_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __strrchr_ia32, .-__strrchr_ia32 -# undef libc_hidden_builtin_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI_strrchr; __GI_strrchr = __strrchr_ia32 -#endif - -#include "../../strrchr.S" diff --git a/sysdeps/i386/i686/multiarch/strrchr.c b/sysdeps/i386/i686/multiarch/strrchr.c new file mode 100644 index 0000000000..ffacd0e625 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strrchr.c @@ -0,0 +1,32 @@ +/* Multiple versions of strrchr. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strrchr __redirect_strrchr +# include <string.h> +# undef strrchr + +# define SYMBOL_NAME strrchr +# include "ifunc-sse2-bsf.h" + +libc_ifunc_redirected (__redirect_strrchr, strrchr, IFUNC_SELECTOR ()); + +weak_alias (strrchr, rindex) +#endif diff --git a/sysdeps/i386/i686/multiarch/strspn-ia32.S b/sysdeps/i386/i686/multiarch/strspn-ia32.S new file mode 100644 index 0000000000..d2b9beb8ee --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strspn-ia32.S @@ -0,0 +1,33 @@ +/* strspn optimized for i686. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +#if IS_IN (libc) +# define strspn __strspn_ia32 + +# ifdef SHARED +# undef libc_hidden_builtin_def +/* IFUNC doesn't work with the hidden functions in shared library since + they will be called without setting up EBX needed for PLT which is + used by IFUNC. */ +# define libc_hidden_builtin_def(name) \ + .globl __GI_strspn; __GI_strspn = strspn +# endif +#endif + +#include <sysdeps/i386/strspn.S> diff --git a/sysdeps/i386/i686/multiarch/strspn.S b/sysdeps/i386/i686/multiarch/strspn.S deleted file mode 100644 index 1269062381..0000000000 --- a/sysdeps/i386/i686/multiarch/strspn.S +++ /dev/null @@ -1,56 +0,0 @@ -/* Multiple versions of strspn - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2009-2017 Free Software Foundation, Inc. - Contributed by Intel Corporation. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - <http://www.gnu.org/licenses/>. */ - -#include <config.h> -#include <sysdep.h> -#include <init-arch.h> - -/* Define multiple versions only for the definition in libc. */ -#if IS_IN (libc) - .text -ENTRY(strspn) - .type strspn, @gnu_indirect_function - LOAD_GOT_AND_RTLD_GLOBAL_RO - LOAD_FUNC_GOT_EAX (__strspn_ia32) - HAS_CPU_FEATURE (SSE4_2) - jz 2f - LOAD_FUNC_GOT_EAX (__strspn_sse42) -2: ret -END(strspn) - -# undef ENTRY -# define ENTRY(name) \ - .type __strspn_ia32, @function; \ - .globl __strspn_ia32; \ - .p2align 4; \ -__strspn_ia32: cfi_startproc; \ - CALL_MCOUNT -# undef END -# define END(name) \ - cfi_endproc; .size __strspn_ia32, .-__strspn_ia32 -# undef libc_hidden_builtin_def -/* IFUNC doesn't work with the hidden functions in shared library since - they will be called without setting up EBX needed for PLT which is - used by IFUNC. */ -# define libc_hidden_builtin_def(name) \ - .globl __GI_strspn; __GI_strspn = __strspn_ia32 -#endif - -#include "../../strspn.S" diff --git a/sysdeps/i386/i686/multiarch/strspn.c b/sysdeps/i386/i686/multiarch/strspn.c new file mode 100644 index 0000000000..e1a86dfe7c --- /dev/null +++ b/sysdeps/i386/i686/multiarch/strspn.c @@ -0,0 +1,30 @@ +/* Multiple versions of strspn. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define strspn __redirect_strspn +# include <string.h> +# undef strspn + +# define SYMBOL_NAME strspn +# include "ifunc-sse4_2.h" + +libc_ifunc_redirected (__redirect_strspn, strspn, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/wcschr.c b/sysdeps/i386/i686/multiarch/wcschr.c new file mode 100644 index 0000000000..2dfd318b33 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/wcschr.c @@ -0,0 +1,33 @@ +/* Multiple versions of wcschr. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define wcschr __redirect_wcschr +# define __wcschr __redirect___wcschr +# include <wchar.h> +# undef wcschr +# undef __wcschr + +# define SYMBOL_NAME wcschr +# include "ifunc-sse2.h" + +libc_ifunc_redirected (__redirect_wcschr, __wcschr, IFUNC_SELECTOR ()); +weak_alias (__wcschr, wcschr); +#endif diff --git a/sysdeps/i386/i686/multiarch/wcscmp.c b/sysdeps/i386/i686/multiarch/wcscmp.c new file mode 100644 index 0000000000..e14bd73985 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/wcscmp.c @@ -0,0 +1,33 @@ +/* Multiple versions of wcscmp. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define wcscmp __redirect_wcscmp +# define __wcscmp __redirect___wcscmp +# include <wchar.h> +# undef wcscmp +# undef __wcscmp + +# define SYMBOL_NAME wcscmp +# include "ifunc-sse2.h" + +libc_ifunc_redirected (__redirect_wcscmp, __wcscmp, IFUNC_SELECTOR ()); +weak_alias (__wcscmp, wcscmp); +#endif diff --git a/sysdeps/i386/i686/multiarch/wcscpy.c b/sysdeps/i386/i686/multiarch/wcscpy.c new file mode 100644 index 0000000000..bc17f369e3 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/wcscpy.c @@ -0,0 +1,44 @@ +/* Multiple versions of wcscpy. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define wcscpy __redirect_wcscpy +# include <wchar.h> +# undef wcscpy + +# define SYMBOL_NAME wcscpy +#include <init-arch.h> + +extern __typeof (REDIRECT_NAME) OPTIMIZE (ia32) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (ssse3) attribute_hidden; + +static inline void * +IFUNC_SELECTOR (void) +{ + const struct cpu_features* cpu_features = __get_cpu_features (); + + if (CPU_FEATURES_CPU_P (cpu_features, SSSE3)) + return OPTIMIZE (ssse3); + + return OPTIMIZE (ia32); +} + +libc_ifunc_redirected (__redirect_wcscpy, wcscpy, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/wcslen.c b/sysdeps/i386/i686/multiarch/wcslen.c new file mode 100644 index 0000000000..4870e648fe --- /dev/null +++ b/sysdeps/i386/i686/multiarch/wcslen.c @@ -0,0 +1,31 @@ +/* Multiple versions of wcslen. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define __wcslen __redirect_wcslen +# include <wchar.h> +# undef __wcslen + +# define SYMBOL_NAME wcslen +# include "ifunc-sse2.h" + +libc_ifunc_redirected (__redirect_wcslen, __wcslen, IFUNC_SELECTOR ()); +weak_alias (__wcslen, wcslen); +#endif diff --git a/sysdeps/i386/i686/multiarch/wcsrchr.c b/sysdeps/i386/i686/multiarch/wcsrchr.c new file mode 100644 index 0000000000..3169febd8f --- /dev/null +++ b/sysdeps/i386/i686/multiarch/wcsrchr.c @@ -0,0 +1,30 @@ +/* Multiple versions of wcsrchr. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define wcsrchr __redirect_wcsrchr +# include <wchar.h> +# undef wcsrchr + +# define SYMBOL_NAME wcsrchr +# include "ifunc-sse2.h" + +libc_ifunc_redirected (__redirect_wcsrchr, wcsrchr, IFUNC_SELECTOR ()); +#endif diff --git a/sysdeps/i386/i686/multiarch/wmemcmp.c b/sysdeps/i386/i686/multiarch/wmemcmp.c new file mode 100644 index 0000000000..ca61408650 --- /dev/null +++ b/sysdeps/i386/i686/multiarch/wmemcmp.c @@ -0,0 +1,30 @@ +/* Multiple versions of wmemcmp. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <http://www.gnu.org/licenses/>. */ + +/* Define multiple versions only for the definition in libc. */ +#if IS_IN (libc) +# define wmemcmp __redirect_wmemcmp +# include <wchar.h> +# undef wmemcmp + +# define SYMBOL_NAME wmemcmp +# include "ifunc-ssse3-sse4_2.h" + +libc_ifunc_redirected (__redirect_wmemcmp, wmemcmp, IFUNC_SELECTOR ()); +#endif |