about summary refs log tree commit diff
Commit message (Collapse)AuthorAgeFilesLines
* math: new log2fSzabolcs Nagy2019-04-173-58/+108
| | | | | | | | | | | | | | | from https://github.com/ARM-software/optimized-routines, commit 04884bd04eac4b251da4026900010ea7d8850edc code size change: +177 bytes. benchmark on x86_64 before, after, speedup: -Os: log2f rthruput: 11.38 ns/call 5.99 ns/call 1.9x log2f latency: 35.01 ns/call 22.57 ns/call 1.55x -O3: log2f rthruput: 10.82 ns/call 5.58 ns/call 1.94x log2f latency: 35.13 ns/call 21.04 ns/call 1.67x
* math: new logfSzabolcs Nagy2019-04-173-54/+109
| | | | | | | | | | | | | | | | from https://github.com/ARM-software/optimized-routines, commit 04884bd04eac4b251da4026900010ea7d8850edc, with minor changes to better fit into musl. code size change: +289 bytes. benchmark on x86_64 before, after, speedup: -Os: logf rthruput: 8.40 ns/call 6.14 ns/call 1.37x logf latency: 31.79 ns/call 24.33 ns/call 1.31x -O3: logf rthruput: 8.43 ns/call 5.58 ns/call 1.51x logf latency: 32.04 ns/call 20.88 ns/call 1.53x
* math: add configuration macrosSzabolcs Nagy2019-04-171-0/+5
| | | | | | | Musl currently aims to support non-nearest rounding mode and does not support SNaNs. These macros allow marking relevant code paths in case these decisions are changed later (they also help documenting the corner cases involved).
* math: add macros for static branch prediction hintsSzabolcs Nagy2019-04-171-0/+9
| | | | | | | | These don't have an effectw with -Os so not useful with default settings other than documenting the expectation. With --enable-optimize=internal,malloc,string,math the libc.so code size increases by 18K on x86_64 and performance varies in -2% .. +10%.
* math: add double precision error handling functionsSzabolcs Nagy2019-04-176-0/+35
|
* math: add single precision error handling functionsSzabolcs Nagy2019-04-176-0/+37
| | | | | | | | | | These are supposed to be used in tail call positions when handling special cases in new code. (fp exceptions may be raised "naturally" by the common code path if special casing is more effort.) This implements the error handling apis used in https://github.com/ARM-software/optimized-routines without errno setting.
* math: add eval_as_float and eval_as_doubleSzabolcs Nagy2019-04-171-0/+17
| | | | | | | | | | | | | | | | | | Previously type casts or assignments were used for handling excess precision, which assumed standard C99 semantics, but since it's a rarely needed obscure detail, it's better to use explicit helper functions to document where we rely on this. It also helps if the code is used outside of the libc in non-C99 compilation mode: with the default excess precision handling of gcc, explicit inline asm barriers are needed for narrowing on FLT_EVAL_METHOD!=0 targets. I plan to use this in new code with the existing style that uses double_t and float_t as much as possible. One ugliness is that it is required for almost every return statement since that does not drop excess precision (the standard changed this in C11 annex F, but that does not help in non-standard compilation modes or with old compilers).
* math: add fp_arch.h with fp_barrier and fp_force_evalSzabolcs Nagy2019-04-173-6/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | C99 has ways to support fenv access, but compilers don't implement it and assume nearest rounding mode and no fp status flag access. (gcc has -frounding-math and then it does not assume nearest rounding mode, but it still assumes the compiled code itself does not change the mode. Even if the C99 mechanism was implemented it is not ideal: it requires all code in the library to be compiled with FENV_ACCESS "on" to make it usable in non-nearest rounding mode, but that limits optimizations more than necessary.) The math functions should give reasonable results in all rounding modes (but the quality may be degraded in non-nearest rounding modes) and the fp status flag settings should follow the spec, so fenv side-effects are important and code transformations that break them should be prevented. Unfortunately compilers don't give any help with this, the best we can do is to add fp barriers to the code using volatile local variables (they create a stack frame and undesirable memory accesses to it) or inline asm (gcc specific, requires target specific fp reg constraints, often creates unnecessary reg moves and multiple barriers are needed to express that an operation has side-effects) or extern call (only useful in tail-call position to avoid stack-frame creation and does not work with lto). We assume that in a math function if an operation depends on the input and the output depends on it, then the operation will be evaluated at runtime when the function is called, producing all the expected fenv side-effects (this is not true in case of lto and in case the operation is evaluated with excess precision that is not rounded away). So fp barriers are needed (1) to prevent the move of an operation within a function (in case it may be moved from an unevaluated code path into an evaluated one or if it may be moved across a fenv access), (2) force the evaluation of an operation for its side-effect when it has no input dependency (may be constant folded) or (3) when its output is unused. I belive that fp_barrier and fp_force_eval can take care of these and they should not be needed in hot code paths.
* math: remove sun copyright from libm.hSzabolcs Nagy2019-04-171-23/+0
| | | | | | | Nothing is left from the original fdlibm header nor from the bsd modifications to it other than some internal api declarations. Comments are dropped that may be copyrightable content.
* math: add asuint, asuint64, asfloat and asdoubleSzabolcs Nagy2019-04-171-33/+15
| | | | | Code generation for SET_HIGH_WORD slightly changes, but it only affects pow, otherwise the generated code is unchanged.
* math: move complex math out of libm.hSzabolcs Nagy2019-04-1767-80/+87
| | | | | | This makes it easier to build musl math code with a compiler that does not support complex types (tcc) and in general more sensible factorization of the internal headers.
* define FP_FAST_FMA* when fma* can be inlinedSzabolcs Nagy2019-04-171-0/+12
| | | | | | | | | | | | | | | | | | | FP_FAST_FMA can be defined if "the fma function generally executes about as fast as, or faster than, a multiply and an add of double operands", which can only be true if the fma call is inlined as an instruction. gcc sets __FP_FAST_FMA if __builtin_fma is inlined as an instruction, but that does not mean an fma call will be inlined (e.g. it is defined with -fno-builtin-fma), other compilers (clang) don't even have such macro, but this is the closest we can get. (even if the libc fma implementation is a single instruction, the extern call overhead is already too big when the macro is used to decide between x*y+z and fma(x,y,z) so it cannot be based on libc only, defining the macro unconditionally on targets which have fma in the base isa is also incorrect: the compiler might not inline fma anyway.) this solution works with gcc unless fma inlining is explicitly turned off.
* fcntl.h: define O_TTY_INIT to 0A. Wilcox2019-04-101-2/+3
| | | | | | | | | | POSIX: "[If] either O_TTY_INIT is set in oflag or O_TTY_INIT has the value zero, open() shall set any non-standard termios structure terminal parameters to a state that provides conforming behavior." The Linux kernel tty drivers always perform initialisation on their devices to set known good termios values during the open(2) call. This means that setting O_TTY_INIT to zero is conforming.
* remove external __syscall function and last remaining usersRich Felker2019-04-1018-264/+2
| | | | | | | | | | | | | | the weak version of __syscall_cp_c was using a tail call to __syscall to avoid duplicating the 6-argument syscall code inline in small static-linked programs, but now that __syscall no longer exists, the inline expansion is no longer duplication. the syscall.h machinery suppported up to 7 syscall arguments, only via an external __syscall function, but we presently have no syscall call points that actually make use of that many, and the kernel only defines 7-argument calling conventions for arm, powerpc (32-bit), and sh. if it turns out we need them in the future, they can easily be added.
* implement inline 5- and 6-argument syscalls for mipsn32 and mips64Rich Felker2019-04-102-29/+68
| | | | | | | | | | n32 and n64 ABIs add new argument registers vs o32, so that passing on the stack is not necessary, so it's not clear why the 5- and 6-argument versions were special-cased to begin with; it seems to have been pattern-copying from arch/mips (o32). i've treated the new argument registers like the first 4 in terms of clobber status (non-clobbered). hopefully this is correct.
* cleanup mips64 syscall_arch functionsRich Felker2019-04-101-18/+9
|
* implement inline 5- and 6-argument syscalls for mipsRich Felker2019-04-101-6/+33
| | | | | | | | the OABI passes these on the stack, using the convention that their position on the stack is as if the first four arguments (in registers) also had stack slots. originally this was deemed too awkward to do inline, falling back to external __syscall, but it's not that bad and now that external __syscall is being removed, it's necessary.
* use inline syscalls for powerpc (32-bit)Rich Felker2019-04-101-2/+84
| | | | | | the inline syscall code is copied directly from powerpc64. the extent of register clobber specifiers may be excessive on both; if that turns out to be the case it can be fixed later.
* remove cruft for supposedly-buggy clang from or1k & microblaze syscall_archRich Felker2019-04-102-18/+0
| | | | | | | | | it was never demonstrated to me that this workaround was needed, and seems likely that, if there ever was any clang version for which it was needed, it's old enough to be unusably buggy in other ways. if it turns out some compilers actually can't do the register allocation right, we'll need to replace this with inline shuffling code, since the external __syscall dependency is being removed.
* overhaul i386 syscall mechanism not to depend on external asm sourceRich Felker2019-04-109-80/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | this is the first part of a series of patches intended to make __syscall fully self-contained in the object file produced using syscall.h, which will make it possible for crt1 code to perform syscalls. the (confusingly named) i386 __vsyscall mechanism, which this commit removes, was introduced before the presence of a valid thread pointer was mandatory; back then the thread pointer was setup lazily only if threads were used. the intent was to be able to perform syscalls using the kernel's fast entry point in the VDSO, which can use the sysenter (Intel) or syscall (AMD) instruction instead of int $128, but without inlining an access to the __syscall global at the point of each syscall, which would incur a significant size cost from PIC setup everywhere. the mechanism also shuffled registers/calling convention around to avoid spills of call-saved registers, and to avoid allocating ebx or ebp via asm constraints, since there are plenty of broken-but-supported compiler versions which are incapable of allocating ebx with -fPIC or ebp with -fno-omit-frame-pointer. the new mechanism preserves the properties of avoiding spills and avoiding allocation of ebx/ebp in constraints, but does it inline, using some fairly simple register shuffling, and uses a field of the thread structure rather than global data for the vdso-provided syscall code address. for now, the external __syscall function is refactored not to use the old __vsyscall so it can be kept, but the intent is to remove it too.
* release 1.1.22 v1.1.22Rich Felker2019-04-092-1/+42
|
* in membarrier fallback, allow for possibility that sigaction failsRich Felker2019-04-091-8/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | this is a workaround to avoid a crashing regression on qemu-user when dynamic TLS is installed at dlopen time. the sigaction syscall should not be able to fail, but it does fail for implementation-internal signals under qemu user-level emulation if the host libc qemu is running under reserves the same signals for implementation-internal use, since qemu makes no provision to redirect/emulate them. after sigaction fails, the subsequent tkill would terminate the process abnormally as the default action. no provision to account for membarrier failing is made in the dynamic linker code that installs new TLS. at the formal level, the missing barrier in this case is incorrect, and perhaps we should fail the dlopen operation, but in practice all the archs we support (and probably all real-world archs except alpha, which isn't yet supported) should give the right behavior with no barrier at all as a consequence of consume-order properties. in the long term, this workaround should be supplemented or replaced by something better -- a different fallback approach to ensuring memory consistency, or dynamic allocation of implementation-internal signals. the latter is appealing in that it would allow cancellation to work under qemu-user too, and would even allow many levels of nested emulation.
* fix the use of syscall result in dl_mmapIlya Matveychikov2019-04-061-1/+1
|
* fix signature of function accepted by makecontextBobby Bingham2019-04-051-1/+1
| | | | | | | | This parameter was incorrectly declared to be a pointer to a function accepting zero parameters. The intent of makecontext is that it is possible to pass integer parameters to the function, so this should have been a pointer to a function accepting an unspecified set of parameters.
* fix unintended global symbols in atanl.cDan Gohman2019-04-031-3/+3
| | | | | | | | Mark atanhi, atanlo, and aT in atanl.c as static, as they're not intended to be part of the public API. These are already static in the LDBL_MANT_DIG == 64 code, so this patch is just making the LDBL_MANT_DIG == 113 code do the same thing.
* use __strchrnul instead of strchr and strlen in execvpeFrediano Ziglio2019-04-021-2/+1
| | | | | | The result is the same but takes less code. Note that __execvpe calls getenv which calls __strchrnul so even using static output the size of the executable won't grow.
* delete a redundant if in dynamic linker ctor execution loopRay2019-04-021-1/+0
|
* fix harmless-by-chance typo in priority inheritance mutex codeRich Felker2019-04-011-1/+1
| | | | | | | commit 54ca677983d47529bab8752315ac1a2b49888870 inadvertently introduced bitwise and where logical and was intended. since the right-hand operand is always 0 or -1 whenever the left-hand operand is nonzero, the behavior happened to be equivalent.
* implement priority inheritance mutexesRich Felker2019-03-314-8/+93
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | priority inheritance is a feature to mitigate priority inversion situations, where a execution of a medium-priority thread can unboundedly block forward progress of a high-priority thread when a lock it needs is held by a low-priority thread. the natural way to do priority inheritance would be with a simple futex flag to donate the calling thread's priority to a target thread while it waits on the futex. unfortunately, linux does not offer such an interface, but instead insists on implementing the whole locking protocol in kernelspace with special futex commands that exist solely for the purpose of doing PI mutexes. this would require the entire "trylock" logic to be duplicated in the timedlock code path for PI mutexes, since, once the previous lock holder releases the lock and the futex call returns, the lock is already held by the caller. obviously such code duplication is undesirable. instead, I've made the PI timedlock success path set the mutex lock count to -1, which can be thought of as "not yet complete", since a lock count of 0 is "locked, with no recursive references". a simple branch in a non-hot path of pthread_mutex_trylock can then see and act on this state, skipping past the code that would check and take the lock to the same code path that runs after the lock is obtained for a non-PI mutex. because we're forced to let the kernel perform the actual lock and unlock operations whenever the mutex is contended, we have to patch things up when it does the wrong thing: 1. the lock operation is not aware of whether the mutex is error-checking, so it will always fail with EDEADLK rather than deadlocking. 2. the lock operation is not aware of whether the mutex is robust, so it will successfully obtain mutexes in the owner-died state even if they're non-robust, whereas this operation should deadlock. 3. the unlock operation always sets the lock value to zero, whereas for robust mutexes, we want to set it to a special value indicating that the mutex obtained after its owner died was unlocked without marking it consistent, so that future operations all fail with ENOTRECOVERABLE. the first of these is easy to solve, just by performing a futex wait on a dummy futex address to simulate deadlock or ETIMEDOUT as appropriate. but problems 2 and 3 interact in a nasty way. to solve problem 2, we need to back out the spurious success. but if waiters are present -- which we can't just ignore, because even if we don't want to wake them, the calling thread is incorrectly inheriting their priorities -- this requires using the kernel's unlock operation, which will zero the lock value, thereby losing the "owner died with lock held" state. to solve these problems, we overload the mutex's waiters field, which is unused for PI mutexes since they don't call the normal futex wait functions, as an indicator that the PI mutex is permanently non-lockable. originally I wanted to use the count field, but there is one code path that needs to access this flag without synchronization: trylock's CAS failure path needs to be able to decide whether to fail with EBUSY or ENOTRECOVERABLE, the waiters field is already treated as a relaxed-order atomic in our memory model, so this works out nicely.
* clean up access to mutex type in pthread_mutex_trylockRich Felker2019-03-291-2/+2
| | | | | | | there was no point in masking off the pshared bit when first loading the type, since every subsequent access involves a mask anyway. not masking it may avoid a subsequent load to check the pshared flag, and it's just simpler.
* support archs with no renameat syscall, only renameat2Drew DeVault2019-03-212-2/+8
|
* support archs with no mlock syscall, only mlock2Drew DeVault2019-03-211-0/+4
|
* fix data race choosing next key slot in pthread_key_createRich Felker2019-03-211-1/+1
| | | | | | | | | commit 84d061d5a31c9c773e29e1e2b1ffe8cb9557bc58 wrongly moved the access to the global next_key outside of the scope of the lock. the error manifested as spurious failure to find an available key slot under concurrent calls to pthread_key_create, since the stopping condition could be met after only a small number of slots were examined.
* fix crash/out-of-bound read in sscanfRich Felker2019-03-142-2/+3
| | | | | | | | | | | | | | | | | | | | | commit d6c855caa88ddb1ab6e24e23a14b1e7baf4ba9c7 caused this "regression", though the behavior was undefined before, overlooking that f->shend=0 was being used as a sentinel for "EOF" status (actual EOF or hitting the scanf field width) of the stream helper (shgetc) functions. obviously the shgetc macro could be adjusted to check for a null pointer in addition to the != comparison, but it's the hot path, and adding extra code/branches to it begins to defeat the purpose. so instead of setting shend to a null pointer to block further reads, which no longer works, set it to the current position (rpos). this makes the shgetc macro work with no change, but it breaks shunget, which can no longer look at the value of shend to determine whether to back up. Szabolcs Nagy suggested a solution which I'm using here: setting shlim to a negative value is inexpensive to test at shunget time, and automatically re-trips the cnt>=shlim stop condition in __shgetc no matter what the original limit was.
* fix namespace violation in dependencies of mtx_lockRich Felker2019-03-131-1/+1
| | | | | | commit 2de29bc994029b903a366b8a4a9f8c3c3ee2be90 left behind one reference to pthread_mutex_trylock. fixing this also improves code generation due to the namespace-safe version being hidde.
* aarch64: add HWCAP_ definitions from linux v5.0Szabolcs Nagy2019-03-131-0/+3
| | | | | | | | HWCAP_SB - speculation barrier instruction available added in linux commit bd4fb6d270bc423a9a4098108784f7f9254c4e6d HWCAP_PACA, HWCAP_PACG - pointer authentication instructions available (address and generic) added in linux commit 7503197562567b57ec14feb3a9d5400ebc56812f
* sys/prctl.h: add PR_PAC_RESET_KEYS from linux v5.0Szabolcs Nagy2019-03-131-0/+7
| | | | | | aarch64 pointer authentication code related prctl that allows reinitializing the key for the thread, added in linux commit ba830885656414101b2f8ca88786524d4bb5e8c1
* elf.h: add NT_ definitions from linux v5.0Szabolcs Nagy2019-03-131-0/+2
| | | | | | | NT_MIPS_MSA for ptrace access to mips simd arch reg set, added in linux commit 3cd640832894b85b5929d5bda74505452c800421 NT_ARM_PAC_MASK for ptrace access to pointer auth code mask, added in commit ec6e822d1a22d0eef1d1fa260dff751dba9a4258
* elf.h: update with C-SKY definitionsSzabolcs Nagy2019-03-131-1/+57
| | | | | | | C-SKY support was added to binutils 2.32 in commit b8891f8d622a31306062065813fc278d8a94fe21 the elf.h change was added to glibc 2.29 in commit 4975f0c3d0131fdf697be0b1631c265e5fd39088
* aarch64, or1k: add kexec_file_load syscall number from linux v5.0Szabolcs Nagy2019-03-132-0/+2
| | | | added in linux commit 4e21565b7fd4d9045765f697887e74a704135fe2
* netinet/tcp.h: add TCP_NLA_SRTT from linux v5.0Szabolcs Nagy2019-03-131-0/+1
| | | | | smoothed RTT for SCM_TIMESTAMPING_OPT_STATS control messages. added in linux commit e8bd8fca6773ef49390269bd467bf940a0841ccf
* netinet/udp.h: add UDP_GRO from linux v5.0Szabolcs Nagy2019-03-131-0/+1
| | | | | sockopt to enable gro for udp. added in linux commit e20cf8d3f1f763ad28a9cb3b41305b8a8a42653e
* powerpc: add PTRACE_SYSEMU from linux v4.20Szabolcs Nagy2019-03-132-0/+4
| | | | added in linux commit 5521eb4bca2db733952f068c37bdf3cd656ad23c
* aarch64: add HWCAP_SSBS from linux v4.20Szabolcs Nagy2019-03-131-0/+1
| | | | | for armv8.5 speculative store bypass PSTATE bit support, added in linux commit d71be2b6c0e19180b5f80a6d42039cc074a693a2
* bits/ioctl.h: add TIOC{G,S}ISO7816 from linux v4.20Szabolcs Nagy2019-03-131-0/+2
| | | | | | | | | | | | | | | | | | | | | ISO7816 smart cards ioctls. linux commit ad8c0eaa0a418ae8ef3f9217638bb86439399eac the actual kernel definitions are #define TIOCGISO7816 _IOR('T', 0x42, struct serial_iso7816) #define TIOCSISO7816 _IOWR('T', 0x43, struct serial_iso7816) where struct serial_iso7816 is defined in linux/serial.h as struct serial_iso7816 { __u32 flags; __u32 tg; __u32 sc_fi; __u32 sc_di; __u32 clk; __u32 reserved[5]; };
* sys/prctl.h: add PR_SPEC_INDIRECT_BRANCH from linux v4.20Szabolcs Nagy2019-03-131-0/+1
| | | | | | prctls to allow per task control of indirect branch speculation on x86. added in linux commit 9137bb27e60e554dab694eafa4cca241fa3a694f
* netinet/in.h add IPV6_MULTICAST_ALL from linux v4.20Szabolcs Nagy2019-03-131-0/+1
| | | | | | ipv6 analogue of IP_MULTICAST_ALL sockopt. added in linux commit 15033f0457dca569b284bef0c8d3ad55fb37eacb
* add PACKET_IGNORE_OUTGOING sockopt from linux v4.20Szabolcs Nagy2019-03-131-0/+1
| | | | new in linux commit fa788d986a3aac5069378ed04697bd06f83d3488
* sys/mman.h: add new hugetlb mmap flags from linux v4.19Szabolcs Nagy2019-03-132-0/+4
| | | | | aarch64 supports 32MB and 512MB hugetlb page sizes too. added in linux commit 20916d4636a9b3c1bf562b305f91d126771edaf9
* arm: add io_pgetevents syscall number from v4.19Szabolcs Nagy2019-03-131-0/+1
| | | | wired up in linux commit 73aeb2cbcdc9be391b3d32a55319a59ce425426f
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-08 02:30:17 +0000