| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
i originally omitted these (optional, per POSIX) interfaces because i
considered them backwards implementation details. however, someone
later brought to my attention a fairly legitimate use case: allocating
thread stacks in memory that's setup for sharing and/or fast transfer
between CPU and GPU so that the thread can move data to a GPU directly
from automatic-storage buffers without having to go through additional
buffer copies.
perhaps there are other situations in which these interfaces are
useful too.
|
| |
|
|
|
|
|
|
| |
this fix is necessary because a program could be started with some of
the implementation-reserved signals masked (e.g. due to exec having
been called from a signal handler, or from a non-musl program) and
then could obtain an invalid-to-use-later sigset_t as the old/saved
signal mask.
|
| |
|
|
|
|
|
| |
this action is now performed in pthread_self initialization; it must
be performed there in case the first call to pthread_create is from a
signal handler, in which case the old signal mask could be restored on
return from the signal.
|
| |
|
|
|
|
|
|
|
|
|
| |
if the process started with these signals blocked, cancellation could
fail or setxid could deadlock. there is no way to globally unblock
them after threads have been created. by unblocking them in the
pthread_self initialization for the main thread, we ensure that
they're unblocked before any other threads are created and also
outside of any signal handler context (sigaction initialized
pthread_self), which is important so that return from a signal handler
won't re-block them.
|
| |
|
|
|
|
|
|
|
|
|
| |
the bug was that cancellation requests which arrived while a
cancellation point was interrupted by a signal handler would not be
acted upon when the signal handler returns. this was because cp_sp was
never set; it's no longer needed or used.
instead, just always re-raise the signal when cancellation was not
acted upon. this wastes a tiny amount of time in the rare case where
it even matters, but it ensures correctness and simplifies the code.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
stale state information indicating that a thread was possibly blocked
at a cancellation point could get left behind if longjmp was used to
exit a signal handler that interrupted a cancellation point.
to fix the issue, we throw away the state information entirely and
simply compare the saved instruction pointer to a range of code
addresses in the __syscall_cp_asm function. all the ugly PIC work
(which becomes minimal anyway with this approach) is defered to
cancellation time instead of happening at every syscall, which should
improve performance too.
this commit also fixes cancellation on arm, which was mildly broken
(race condition, not checking cancellation flag once inside the
cancellation point zone). apparently i forgot to implement that. the
new arm code is untested, but appears correct; i'll test and fix it
later if there are problems.
|
| |
|
|
|
| |
no need to pass unnecessary extra arguments on to the core code in
pthread_create.c. this just wastes cycles and code bloat.
|
| |
|
|
|
|
|
|
|
|
| |
i made a best attempt, but the intended semantics of this function are
fundamentally contradictory. there is no consistent way to handle
ownership of locks when forking a multi-threaded process. the code
could have worked by accident for programs that only used normal
mutexes and nothing else (since they don't actually store or care
about their owner), but that's about it. broken-by-design interfaces
that aren't even in glibc (only solaris) don't belong in musl.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
these changes are based on the following communication via email:
"I hereby grant that all of the code I have contributed to musl on or
before April 23, 2012 may be licensed under the terms of the following
MIT license:
Copyright (c) 2011-2012 Nicholas J. Kain
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
this change is necessary or pthread_create will always fail on
security-hardened kernels. i considered first trying to make the stack
executable and simply retrying without execute permissions when the
first try fails, but (1) this would incur a serious performance
penalty on hardened systems, and (2) having the stack be executable is
just a bad idea from a security standpoint.
if there is real-world "GNU C" code that uses nested functions with
threads, and it can't be fixed, we'll have to consider other ways of
solving the problem, but for now this seems like the best fix.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
pthread structure has been adjusted to match the glibc/GCC abi for
where the canary is stored on i386 and x86_64. it will need variants
for other archs to provide the added security of the canary's entropy,
but even without that it still works as well as the old "minimal" ssp
support. eventually such changes will be made anyway, since they are
also needed for GCC/C11 thread-local storage support (not yet
implemented).
care is taken not to attempt initializing the thread pointer unless
the program actually uses SSP (by reference to __stack_chk_fail).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
i did some testing trying to switch malloc to use the new internal
lock with priority inheritance, and my malloc contention test got
20-100 times slower. if priority inheritance futexes are this slow,
it's simply too high a price to pay for avoiding priority inversion.
maybe we can consider them somewhere down the road once the kernel
folks get their act together on this (and perferably don't link it to
glibc's inefficient lock API)...
as such, i've switch __lock to use malloc's implementation of
lightweight locks, and updated all the users of the code to use an
array with a waiter count for their locks. this should give optimal
performance in the vast majority of cases, and it's simple.
malloc is still using its own internal copy of the lock code because
it seems to yield measurably better performance with -O3 when it's
inlined (20% or more difference in the contention stress test).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
this bug probably would have gone unnoticed since it's only used in
the fallback code for systems where priority-inheritance locking
fails. unfortunately this approach results in one spurious wake
syscall on the final unlock, when there are no waiters remaining. the
alternative (possibly better) would be to use broadcast wakes instead
of reflagging the waiter unconditionally, and let each waiter reflag
itself; this saves one syscall at the expense of invoking the
"thundering herd" effect (worse performance degredation) when there
are many waiters.
ideally we would be able to update all of our locks to use an array of
two ints rather than a single int, and use a separate counter system
like proper mutexes use; then we could avoid all spurious wake calls
without resorting to broadcasts. however, it's not clear to me that
priority inheritance futexes support this usage. the kernel sets the
waiters flag for them (just like we're doing now) and i can't tell if
it's safe to bypass the kernel when unlocking just because we know
(from private data, the waiter count) that there are no waiters. this
is something that could be explored in the future.
|
| |
|
|
|
|
| |
we use priority inheritance futexes if possible so that the library
cannot hit internal priority inversion deadlocks in the presence of
realtime priority scheduling (full support to be added later).
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
this was discussed on the mailing list and no consensus on the
preferred solution was reached, so in anticipation of a release, i'm
just committing a minimally-invasive solution that avoids the problem
by ensuring that multi-threaded-capable programs will always have
initialized the thread pointer before any signal handler can run.
in the long term we may switch to initializing the thread pointer at
program start time whenever the program has the potential to access
any per-thread data.
|
| |
|
|
|
|
|
| |
even if pthread_create/exit code is not linked, run flag needs to be
checked and cleanup function potentially run on pop. thus, move the
code to the module that's always linked when pthread_cleanup_push/pop
is used.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
the old abi was intended to duplicate glibc's abi at the expense of
being ugly and slow, but it turns out glib was not even using that abi
except on non-gcc-compatible compilers (which it doesn't even support)
and was instead using an exceptions-in-c/unwind-based approach whose
abi we could not duplicate anyway without nasty dwarf2/unwind
integration.
the new abi is copied from a very old glibc abi, which seems to still
be supported/present in current glibc. it avoids all unwinding,
whether by sjlj or exceptions, and merely maintains a linked list of
cleanup functions to be called from the context of pthread_exit. i've
made some care to ensure that longjmp out of a cleanup function should
work, even though it is not required to.
this change breaks abi compatibility with programs which were using
pthread cancellation, which is unfortunate, but that's why i'm making
the change now rather than later. considering that most pthread
features have not been usable until recently anyway, i don't see it as
a major issue at this point.
|
| |
|
|
|
|
|
| |
even a single-threaded program can be cancellable, e.g. if it's called
pthread_cancel(pthread_self()). the correct predicate to check is not
whether multiple threads have been invoked, but whether pthread_self
has been invoked.
|
| |
|
|
|
| |
this is not required by the standard, but it's nicer than corrupting
the state and rather inexpensive.
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
right now it's questionable whether this change is an improvement or
not, but if we later want to support priority inheritance mutexes, it
will be important to have the code paths unified like this to avoid
major code duplication.
|
| |
|
|
|
| |
this is valid for error-checking mutexes; otherwise it invokes UB and
would be justified in crashing.
|
| |
|
|
|
|
|
| |
this simplifies the code paths slightly, but perhaps what's nicer is
that it makes recursive mutexes fully reentrant, i.e. locking and
unlocking from a signal handler works even if the interrupted code was
in the middle of locking or unlocking.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
a reader unlocking the lock need only wake one waiter (necessarily a
writer, but a writer unlocking the lock must wake all waiters
(necessarily readers). if it only wakes one, the remainder can remain
blocked indefinitely, or at least until the first reader unlocks (in
which case the whole lock becomes serialized and behaves as a mutex
rather than a read lock).
|
| |
|
|
|
|
| |
there is no need to send a wake when the lock count does not hit zero,
but when it does, all waiters must be woken (since all with the same
sign are eligible to obtain the lock).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
eliminate the sequence number field and instead use the counter as the
futex because of the way the lock is held, sequence numbers are
completely useless, and this frees up a field in the barrier structure
to be used as a waiter count for the count futex, which lets us avoid
some syscalls in the best case.
as of now, self-synchronized destruction and unmapping should be fully
safe. before any thread can return from the barrier, all threads in
the barrier have obtained the vm lock, and each holds a shared lock on
the barrier. the barrier memory is not inspected after the shared lock
count reaches 0, nor after the vm lock is released.
|
| |
|
|
| |
i think this works, but it can be simplified. (next step)
|
| |
|
|
|
| |
the vm lock only waits for threads in the same process exiting.
actually this fix is not enough, but it's a start...
|
| | |
|
| |
|
|
|
|
| |
it was assuming the result of the condition it was supposed to be
checking for, i.e. that the thread ptr had already been initialized by
pthread_mutex_lock. use the slower call to be safe.
|
| | |
|
| |
|
|
|
|
|
| |
we're not required to check this except for error-checking mutexes,
but it doesn't hurt. the new test is actually simpler/lighter, and it
also eliminates the need to later check that pthread_mutex_unlock
succeeds.
|
| |
|
|
|
|
|
|
|
|
| |
when used with error-checking mutexes, pthread_cond_wait is required
to fail with EPERM if the mutex is not locked by the caller.
previously we relied on pthread_mutex_unlock to generate the error,
but this is not valid, since in the case of such invalid usage the
internal state of the cond variable has already been potentially
corrupted (due to access outside the control of the mutex). thus, we
have to check first.
|
| |
|
|
| |
i set the return value but then never used it... oops!
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
this implementation is rather heavy-weight, but it's the first
solution i've found that's actually correct. all waiters actually wait
twice at the barrier so that they can synchronize exit, and they hold
a "vm lock" that prevents changes to virtual memory mappings (and
blocks pthread_barrier_destroy) until all waiters are finished
inspecting the barrier.
thus, it is safe for any thread to destroy and/or unmap the barrier's
memory as soon as pthread_barrier_wait returns, without further
synchronization.
|
| |
|
|
|
| |
mmap returns MAP_FAILED not 0 because some idiot thought the ability
to mmap the null pointer page would be a good idea...
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
lock out new waiters during the broadcast. otherwise the wait count
added to the mutex might be lower than the actual number of waiters
moved, and wakeups may be lost.
this issue could also be solved by temporarily setting the mutex
waiter count higher than any possible real count, then relying on the
kernel to tell us how many waiters were requeued, and updating the
counts afterwards. however the logic is more complex, and i don't
really trust the kernel. the solution here is also nice in that it
replaces some atomic cas loops with simple non-atomic ops under lock.
|
| |
|
|
|
|
|
|
|
|
|
| |
due to moving waiters from the cond var to the mutex in bcast, these
waiters upon wakeup would steal slots in the count from newer waiters
that had not yet been signaled, preventing the signal function from
taking any action.
to solve the problem, we simply use two separate waiter counts, and so
that the original "total" waiters count is undisturbed by broadcast
and still available for signal.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
testing revealed that the old implementation, while correct, was
giving way too many spurious wakeups due to races changing the value
of the condition futex. in a test program with 5 threads receiving
broadcast signals, the number of returns from pthread_cond_wait was
roughly 3 times what it should have been (2 spurious wakeups for every
legitimate wakeup). moreover, the magnitude of this effect seems to
grow with the number of threads.
the old implementation may also have had some nasty race conditions
with reuse of the cond var with a new mutex.
the new implementation is based on incrementing a sequence number with
each signal event. this sequence number has nothing to do with the
number of threads intended to be woken; it's only used to provide a
value for the futex wait to avoid deadlock. in theory there is a
danger of race conditions due to the value wrapping around after 2^32
signals. it would be nice to eliminate that, if there's a way.
testing showed no spurious wakeups (though they are of course
possible) with the new implementation, as well as slightly improved
performance.
|
| |
|
|
|
|
| |
using swap has a race condition: the waiters must be added to the
mutex waiter count *before* they are taken off the cond var waiter
count, or wake events can be lost.
|
| | |
|
| |
|
|
| |
somehow i forgot that normal-type mutexes don't store the owner tid.
|
| |
|
|
|
|
| |
this avoids the "stampede effect" where pthread_cond_broadcast would
result in all waiters waking up simultaneously, only to immediately
contend for the mutex and go back to sleep.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
previously, a waiter could miss the 1->0 transition of block if
another thread set block to 1 again after the signal function set
block to 0. we now use the caller's thread id as a unique token to
store in block, which no other thread will ever write there. this
ensures that if block still contains the tid, no signal has occurred.
spurious wakeups will of course occur whenever there is a spurious
return from the futex wait and another thread has begun waiting on the
cond var. this should be a rare occurrence except perhaps in the
presence of interrupting signal handlers.
signal/bcast operations have been improved by noting that they need
not avoid inspecting the cond var's memory after changing the futex
value. because the standard allows spurious wakeups, there is no way
for an application to distinguish between a spurious wakeup just
before another thread called signal/bcast, and the deliberate wakeup
resulting from the signal/bcast call. thus the woken thread must
assume that the signalling thread may still be waiting to act on the
cond var, and therefore it cannot destroy/unmap the cond var.
|
| |
|
|
|
| |
it's amazing none of the conformance tests i've run even bothered to
check whether something so basic works...
|
