redesign cond var implementation to fix multiple issues

the immediate issue that was reported by Jens Gustedt and needed to be
fixed was corruption of the cv/mutex waiter states when switching to
using a new mutex with the cv after all waiters were unblocked but
before they finished returning from the wait function.

self-synchronized destruction was also handled poorly and may have had
race conditions. and the use of sequence numbers for waking waiters
admitted a theoretical missed-wakeup if the sequence number wrapped
through the full 32-bit space.

the new implementation is largely documented in the comments in the
source. the basic principle is to use linked lists initially attached
to the cv object, but detachable on signal/broadcast, made up of nodes
residing in automatic storage (stack) on the threads that are waiting.
this eliminates the need for waiters to access the cv object after
they are signaled, and allows us to limit wakeup to one waiter at a
time during broadcasts even when futex requeue cannot be used.

performance is also greatly improved, roughly double some tests.

basically nothing is changed in the process-shared cond var case,
where this implementation does not work, since processes do not have
access to one another's local storage.

1 files changed, 4 insertions, 35 deletions

diff --git a/src/thread/pthread_cond_broadcast.c b/src/thread/pthread_cond_broadcast.c
index 18e778f3..69f840fb 100644
--- a/src/thread/pthread_cond_broadcast.c
+++ b/src/thread/pthread_cond_broadcast.c
@@ -1,43 +1,12 @@
 #include "pthread_impl.h"
 
+int __private_cond_signal(pthread_cond_t *, int);
+
 int pthread_cond_broadcast(pthread_cond_t *c)
 {
-	pthread_mutex_t *m;
-
+	if (!c->_c_shared) return __private_cond_signal(c, -1);
 	if (!c->_c_waiters) return 0;
-
 	a_inc(&c->_c_seq);
-
-	/* If cond var is process-shared, simply wake all waiters. */
-	if (c->_c_mutex == (void *)-1) {
-		__wake(&c->_c_seq, -1, 0);
-		return 0;
-	}
-
-	/* Block waiters from returning so we can use the mutex. */
-	while (a_swap(&c->_c_lock, 1))
-		__wait(&c->_c_lock, &c->_c_lockwait, 1, 1);
-	if (!c->_c_waiters)
-		goto out;
-	m = c->_c_mutex;
-
-	/* Move waiter count to the mutex */
-	a_fetch_add(&m->_m_waiters, c->_c_waiters2);
-	c->_c_waiters2 = 0;
-
-	/* Perform the futex requeue, waking one waiter unless we know
-	 * that the calling thread holds the mutex. */
-	int wake_cnt = !(m->_m_type & 3)
-		|| (m->_m_lock&INT_MAX)!=__pthread_self()->tid;
-	if (m->_m_type & 128) wake_cnt = INT_MAX;
-	__syscall(SYS_futex, &c->_c_seq, FUTEX_REQUEUE | 128,
-		wake_cnt, INT_MAX, &m->_m_lock) != -EINVAL ||
-	__syscall(SYS_futex, &c->_c_seq, FUTEX_REQUEUE,
-		wake_cnt, INT_MAX, &m->_m_lock);
-
-out:
-	a_store(&c->_c_lock, 0);
-	if (c->_c_lockwait) __wake(&c->_c_lock, 1, 1);
-
+	__wake(&c->_c_seq, -1, 0);
 	return 0;
 }