diff options
author | Adhemerval Zanella <adhemerval.zanella@linaro.org> | 2024-03-05 14:02:57 -0300 |
---|---|---|
committer | Adhemerval Zanella <adhemerval.zanella@linaro.org> | 2024-03-12 14:38:08 -0300 |
commit | 2149da36836bc32cd66359ca37bab5884af7e81f (patch) | |
tree | a238d5f1dfcacdd229f9c4c3d479e8d304195116 /sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c | |
parent | 2173173d57971d042c0ad4b281431ae127e9b5b8 (diff) | |
download | glibc-2149da36836bc32cd66359ca37bab5884af7e81f.tar.gz glibc-2149da36836bc32cd66359ca37bab5884af7e81f.tar.xz glibc-2149da36836bc32cd66359ca37bab5884af7e81f.zip |
riscv: Fix alignment-ignorant memcpy implementation
The memcpy optimization (commit 587a1290a1af7bee6db) has a series of mistakes: - The implementation is wrong: the chunk size calculation is wrong leading to invalid memory access. - It adds ifunc supports as default, so --disable-multi-arch does not work as expected for riscv. - It mixes Linux files (memcpy ifunc selection which requires the vDSO/syscall mechanism) with generic support (the memcpy optimization itself). - There is no __libc_ifunc_impl_list, which makes testing only check the selected implementation instead of all supported by the system. This patch also simplifies the required bits to enable ifunc: there is no need to memcopy.h; nor to add Linux-specific files. The __memcpy_noalignment tail handling now uses a branchless strategy similar to aarch64 (overlap 32-bits copies for sizes 4..7 and byte copies for size 1..3). Checked on riscv64 and riscv32 by explicitly enabling the function on __libc_ifunc_impl_list on qemu-system. Changes from v1: * Implement the memcpy in assembly to correctly handle RISCV strict-alignment. Reviewed-by: Evan Green <evan@rivosinc.com> Acked-by: Palmer Dabbelt <palmer@rivosinc.com>
Diffstat (limited to 'sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c')
-rw-r--r-- | sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c | 43 |
1 files changed, 43 insertions, 0 deletions
diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c new file mode 100644 index 0000000000..9f806d7a9e --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c @@ -0,0 +1,43 @@ +/* Enumerate available IFUNC implementations of a function. RISCV version. + Copyright (C) 2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <ifunc-impl-list.h> +#include <string.h> +#include <sys/hwprobe.h> + +size_t +__libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, + size_t max) +{ + size_t i = max; + + bool fast_unaligned = false; + + struct riscv_hwprobe pair = { .key = RISCV_HWPROBE_KEY_CPUPERF_0 }; + if (__riscv_hwprobe (&pair, 1, 0, NULL, 0) == 0 + && (pair.value & RISCV_HWPROBE_MISALIGNED_MASK) + == RISCV_HWPROBE_MISALIGNED_FAST) + fast_unaligned = true; + + IFUNC_IMPL (i, name, memcpy, + IFUNC_IMPL_ADD (array, i, memcpy, fast_unaligned, + __memcpy_noalignment) + IFUNC_IMPL_ADD (array, i, memcpy, 1, __memcpy_generic)) + + return 0; +} |