| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Current `non_temporal_threshold` set to roughly '3/4 * sizeof_L3 /
ncores_per_socket'. This patch updates that value to roughly
'sizeof_L3 / 4`
The original value (specifically dividing the `ncores_per_socket`) was
done to limit the amount of other threads' data a `memcpy`/`memset`
could evict.
Dividing by 'ncores_per_socket', however leads to exceedingly low
non-temporal thresholds and leads to using non-temporal stores in
cases where REP MOVSB is multiple times faster.
Furthermore, non-temporal stores are written directly to main memory
so using it at a size much smaller than L3 can place soon to be
accessed data much further away than it otherwise could be. As well,
modern machines are able to detect streaming patterns (especially if
REP MOVSB is used) and provide LRU hints to the memory subsystem. This
in affect caps the total amount of eviction at 1/cache_associativity,
far below meaningfully thrashing the entire cache.
As best I can tell, the benchmarks that lead this small threshold
where done comparing non-temporal stores versus standard cacheable
stores. A better comparison (linked below) is to be REP MOVSB which,
on the measure systems, is nearly 2x faster than non-temporal stores
at the low-end of the previous threshold, and within 10% for over
100MB copies (well past even the current threshold). In cases with a
low number of threads competing for bandwidth, REP MOVSB is ~2x faster
up to `sizeof_L3`.
The divisor of `4` is a somewhat arbitrary value. From benchmarks it
seems Skylake and Icelake both prefer a divisor of `2`, but older CPUs
such as Broadwell prefer something closer to `8`. This patch is meant
to be followed up by another one to make the divisor cpu-specific, but
in the meantime (and for easier backporting), this patch settles on
`4` as a middle-ground.
Benchmarks comparing non-temporal stores, REP MOVSB, and cacheable
stores where done using:
https://github.com/goldsteinn/memcpy-nt-benchmarks
Sheets results (also available in pdf on the github):
https://docs.google.com/spreadsheets/d/e/2PACX-1vS183r0rW_jRX6tG_E90m9qVuFiMbRIJvi5VAE8yYOvEOIEEc3aSNuEsrFbuXw5c3nGboxMmrupZD7K/pubhtml
Reviewed-by: DJ Delorie <dj@redhat.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
|
|
|
|
|
|
| |
Previously, the exit function was used, but this causes the test to
block (until the timeout) once exit is changed to lock stdio streams
during flush.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Container management default seccomp filter [1] only accepts
personality(2) with PER_LINUX, (0x0), UNAME26 (0x20000),
PER_LINUX32 (0x8), UNAME26 | PER_LINUX32, and 0xffffffff (to query
current personality)
Although the documentation only state it is blocked to prevent
'enabling BSD emulation' (PER_BSD, not implemented by Linux), checking
on repository log the real reason is to block ASLR disable flag
(ADDR_NO_RANDOMIZE) and other poorly support emulations.
So handle EPERM and fail as UNSUPPORTED if we can really check for
BZ#19408.
Checked on aarch64-linux-gnu.
[1] https://github.com/moby/moby/blob/master/profiles/seccomp/default.json
Reviewed-by: Florian Weimer <fweimer@redhat.com>
|
|
|
|
|
|
|
|
|
| |
As suggested in
<https://sourceware.org/pipermail/libc-alpha/2023-February/145890.html>,
remove the MAP_VARIABLE define from the hppa bits/mman.h, for
consistency with Linux 6.2 which removed the define there.
Tested with build-many-glibcs.py for hppa-linux-gnu.
|
|
|
|
|
|
|
|
|
|
| |
Since the area of the user's stack we use for the registers dump (and
otherwise as __sigreturn2's stack) can and does overlap the sigcontext,
we have to be very careful about the order of loads and stores that we
do. In particular we have to load sc_reply_port before we start
clobbering the sigcontext.
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
|
| |
|
|
|
|
|
| |
Signed-off-by: Alejandro Colomar <alx@kernel.org>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
|
|
|
|
|
|
|
|
|
| |
epoll_pwait2(2)'s second argument should be nonnull. We're going to add
__nonnull to the prototype, so let's fix the test accordingly. We can
use a dummy variable to avoid passing NULL.
Reported-by: Adhemerval Zanella Netto <adhemerval.zanella@linaro.org>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
|
|
|
|
|
|
| |
Use a scratch_buffer rather than alloca to avoid potential stack
overflows.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
|
|
|
|
|
|
| |
Use a scratch_buffer rather than alloca to avoid potential stack
overflows.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
|
|
|
|
|
|
| |
With fortification enabled, few function calls return result need to be
checked, has they get the __wur macro enabled.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
|
|
|
|
|
|
|
| |
Using write without cheks leads to warn unused result when __wur is
enabled.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
|
|
|
|
|
|
|
| |
Since the assembly source file with -evex suffix should use YMM registers,
not ZMM registers, include x86-evex256-vecs.h by default to use YMM
registers in memcmpeq-evex.S
Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Different than other 64 bit architectures, powerpc64 defines the
LFS POSIX lock constants with values similar to 32 ABI, which
are meant to be used with fcntl64 syscall. Since powerpc64 kABI
does not have fcntl, the constants are adjusted with the
FCNTL_ADJUST_CMD macro.
The 4d0fe291aed3a476a changed the logic of generic constants
LFS value are equal to the default values; which is now wrong
for powerpc64.
Fix the value by explicit define the previous glibc constants
(powerpc64 does not need to use the 32 kABI value, but it simplifies
the FCNTL_ADJUST_CMD which should be kept as compatibility).
Checked on powerpc64-linux-gnu and powerpc-linux-gnu.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(BZ#30477)
For architecture with default 64 bit time_t support, the kernel
does not provide LFS and non-LFS values for F_GETLK, F_GETLK, and
F_GETLK (the default value used for 64 bit architecture are used).
This is might be considered an ABI break, but the currenct exported
values is bogus anyway.
The POSIX lockf is not affected since it is aliased to lockf64,
which already uses the LFS values.
Checked on i686-linux-gnu and the new tests on a riscv32.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
|
|
|
|
|
|
|
|
|
| |
The LoongArch glibc was using the value of the SHMLBA macro from common code,
which is __getpagesize() (16k), but this was inconsistent with the value of
the SHMLBA macro in the kernel, which is SZ_64K (64k). This caused several
shmat-related tests in LTP (Linux Test Project) to fail. This commit fixes
the issue by ensuring that the glibc's SHMLBA macro value matches the value
used in the kernel like other architectures.
|
|
|
|
|
|
|
|
|
|
|
| |
It follows the internal signature:
extern int clone3 (struct clone_args *__cl_args, size_t __size,
int (*__func) (void *__arg), void *__arg);
Checked on riscv64-linux-gnu-rv64imafdc-lp64d.
Reviewed-by: Palmer Dabbelt <palmer@rivosinc.com>
|
|
|
|
|
| |
Signed-off-by: Dridi Boukelmoune <dridi.boukelmoune@gmail.com>
Reviewed-by: Arjun Shankar <arjun@redhat.com>
|
|
|
|
|
|
|
|
| |
Use a scratch_buffer rather than either alloca or malloc to reduce the
possibility of a stack overflow.
Suggested-by: Adhemerval Zanella Netto <adhemerval.zanella@linaro.org>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
If `non_temporal_threshold` is below `minimum_non_temporal_threshold`,
it almost certainly means we failed to read the systems cache info.
In this case, rather than defaulting the minimum correct value, we
should default to a value that gets at least reasonable
performance. 64MB is chosen conservatively to be at the very high
end. This should never cause non-temporal stores when, if we had read
cache info, we wouldn't have otherwise.
Reviewed-by: Florian Weimer <fweimer@redhat.com>
|
|
|
|
|
| |
x86_64 needs special alignment when calling functions, so we have to use
MACHINE_THREAD_STATE_SETUP_CALL for the signal thread when forking.
|
|
|
|
|
|
|
|
|
| |
Linux 6.3 adds new constants MFD_NOEXEC_SEAL and MFD_EXEC. Add these
to bits/mman-shared.h (conditional on MFD_NOEXEC_SEAL not already
being defined, similar to the existing conditional on the older MFD_*
macros).
Tested for x86_64.
|
|
|
|
|
|
|
| |
Linux 6.3 adds a new constant IP_LOCAL_PORT_RANGE. Add it to the
corresponding bits/in.h in glibc.
Tested for x86_64.
|
|
|
|
|
|
|
| |
Use a scratch_buffer rather than alloca/malloc to avoid potential stack
overflow.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
|
|
|
|
|
|
|
| |
Fix unused result warnings, detected when _FORTIFY_SOURCE is enabled in
glibc.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
|
|
|
|
|
|
|
| |
In commit 0b25c28e028b63c95108c442d8112811107e4c13 I updated congure.ac
but neglected to regenerate updated configure.
Fix this here.
|
|
|
|
|
|
|
| |
All fixes are in comments, so the binaries should be identical
before/after this commit, but I can't verify this.
Reviewed-by: Rajalakshmi Srinivasaraghavan <rajis@linux.ibm.com>
|
|
|
|
|
|
|
| |
Applying this commit results in bit-identical rebuild of
libc.so.6 math/libm.so.6 elf/ld-linux-x86-64.so.2 mathvec/libmvec.so.1
Reviewed-by: Florian Weimer <fweimer@redhat.com>
|
|
|
|
|
|
|
| |
Applying this commit results in bit-identical rebuild of libc.so.6
math/libm.so.6 elf/ld-linux-x86-64.so.2 mathvec/libmvec.so.1
Reviewed-by: Florian Weimer <fweimer@redhat.com>
|
|
|
|
| |
Fixes: 196358ae26aa ("mach: Fix installing mach_i386.h")
|
|
|
|
|
|
|
| |
Applying this commit results in a bit-identical rebuild of
mathvec/libmvec.so.1 (which is the only binary that gets rebuilt).
Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
|
|
|
|
|
|
| |
We do not want mach_i386.h to get installed into machine/, but into
i386/ or x86_64/ depending where mach_i386.defs was found, i.e.
according to 32/64 bitness.
|
|
|
|
| |
We need O_EXEC for __rtld_execve
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Some of the s390-specific configure checks are using compile and
link configure tests. Now use only compile tests as the link
tests fails when e.g. bootstrapping a cross-toolchain due to
missing crt-files/libc.so. This is achieved by using
AC_COMPILE_IFELSE in configure.ac file.
This is observable e.g. when using buildroot which builds glibc
only once or the build-many-glibcs.py script. Note that the latter
one is building glibc twice in the compilers-step (configure-checks
fails) and in the glibcs-step (configure-checks succeed).
Note, that the s390 specific configure tests for static PIE have to
link an executable to test binutils support. Thus we can't fix
those tests.
|
|
|
|
|
| |
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
Message-Id: <20230520115531.3911877-2-bugaevc@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The __hurd_fail () inline function is the dedicated, idiomatic way of
reporting errors in the Hurd part of glibc. Not only is it more concise
than '{ errno = err; return -1; }', it is since commit
6639cc10029e24e06b34e169712b21c31b8cf213
"hurd: Mark error functions as __COLD" marked with the cold attribute,
telling the compiler that this codepath is unlikely to be executed.
In one case, use __hurd_dfail () over the plain __hurd_fail ().
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
Message-Id: <20230520115531.3911877-1-bugaevc@gmail.com>
|
|
|
|
|
| |
Removed -mabi=ieeelongdouble on failing tests. It resolves the error.
error: ‘-mabi=ieeelongdouble’ requires ‘-mlong-double-128’
|
|
|
|
|
|
|
|
|
|
|
|
| |
Create a private hidden __hurd_thread_self alias, and use that one.
Fixes 2f8ecb58a59eb82c43214d000842d99644a662d1
"hurd: Fix x86_64 _hurd_tls_fork" and
c7fcce38c83a2bb665ef5dc4981bf20c7e586123
"hurd: Make sure to not use tcb->self"
Reported-by: Joseph Myers <joseph@codesourcery.com>
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
|
|
|
|
|
|
|
| |
We had sizeof (time_t) == 8, but __TIMESIZE == 32.
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
Message-Id: <20230519171516.3698754-1-bugaevc@gmail.com>
|
|
|
|
|
| |
90604f670c10 ("hurd 64bit: Add data for check-c++-types") actually added
the 32bit version. This fixes it into a 64bit version.
|
|
|
|
|
|
|
| |
Linux 6.3 adds six HWCAP2_SME* constants for AArch64; add them to the
corresponding bits/hwcap.h in glibc.
Tested with build-many-glibcs.py for aarch64-linux-gnu.
|
|
|
|
|
|
|
|
|
|
|
|
| |
strlen, which is another ifunc-selected function, is invoked during
early static executable startup if the argv arrives from the exec
server. Make it not crash.
Checked on x86_64-gnu: statically linked executables launched after the
exec server is up now start up successfully.
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
Message-Id: <20230517191436.73636-10-bugaevc@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On x86_64, we have to pass function arguments in registers, not on the
stack. We also have to align the stack pointer in a specific way. Since
sharing the logic with i386 does not bring much benefit, split the file
back into i386- and x86_64-specific versions, and fix the x86_64 version
to set up the thread properly.
Bonus: i386 keeps doing the extra RPC inside __thread_set_pcsptp to
fetch the state of the thread before setting it; but x86_64 no lnoger
does that.
Checked on x86_64-gnu and i686-gnu.
Fixes be6d002ca277ffc90058d382396150cb0e785b9c
"hurd: Set up the basic tree for x86_64-gnu"
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
Message-Id: <20230517191436.73636-9-bugaevc@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It is illegal to call thread_get_state () on mach_thread_self (), so
this codepath cannot be used as-is to fork the calling thread's TLS.
Fortunately we can use THREAD_SELF (aka %fs:0x0) to find out the value
of our fs_base without calling into the kernel.
Fixes: f6cf701efc61c9ad910372bda14b9a235db310a8
"hurd: Implement TLS for x86_64"
Checked on x86_64-gnu: fork () now works!
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
Message-Id: <20230517191436.73636-8-bugaevc@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Unlike sigstate->thread, tcb->self did not hold a Mach port reference on
the thread port it names. This means that the port can be deallocated,
and the name reused for something else, without anyone noticing. Using
tcb->self will then lead to port use-after-free.
Fortunately nothing was accessing tcb->self, other than it being
intially set to then-valid thread port name upon TCB initialization. To
assert that this keeps being the case without altering TCB layout,
rename self -> self_do_not_use, and stop initializing it.
Also, do not (re-)allocate a whole separate and unused stack for the
main thread, and just exit __pthread_setup early in this case.
Found upon attempting to use tcb->self and getting unexpected crashes.
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
Message-Id: <20230517191436.73636-7-bugaevc@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
| |
...instead of mach_setup_thread (), which is unsuitable for setting up
function calls.
Checked on x86_64-gnu: the signal thread no longer crashes upon trying
to process a message.
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
Message-Id: <20230517191436.73636-6-bugaevc@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The existing two macros, MACHINE_THREAD_STATE_SET_PC and
MACHINE_THREAD_STATE_SET_SP, can be used to set program counter and the
stack pointer registers in a machine-specific thread state structure.
Useful as it is, this may not be enough to set up the thread to make a
function call, because the machine-specific ABI may impose additional
requirements. In particular, x86_64 ABI requires that upon function
entry, the stack pointer is 8 less than 16-byte aligned (sp & 15 == 8).
To deal with this, introduce a new macro,
MACHINE_THREAD_STATE_SETUP_CALL (), which sets both stack and
instruction pointers, and also applies any machine-specific requirements
to make a valid function call. The default implementation simply
forwards to MACHINE_THREAD_STATE_SET_PC and MACHINE_THREAD_STATE_SET_SP,
but on x86_64 we additionally align the stack pointer.
Signed-off-by: Sergey Bugaev <bugaevc@gmail.com>
Message-Id: <20230517191436.73636-3-bugaevc@gmail.com>
|