overhaul cancellation to fix resource leaks and dangerous behavior with signals

this commit addresses two issues:

1. a race condition, whereby a cancellation request occurring after a
syscall returned from kernelspace but before the subsequent
CANCELPT_END would cause cancellable resource-allocating syscalls
(like open) to leak resources.

2. signal handlers invoked while the thread was blocked at a
cancellation point behaved as if asynchronous cancellation mode wer in
effect, resulting in potentially dangerous state corruption if a
cancellation request occurs.

the glibc/nptl implementation of threads shares both of these issues.

with this commit, both are fixed. however, cancellation points
encountered in a signal handler will not be acted upon if the signal
was received while the thread was already at a cancellation point.
they will of course be acted upon after the signal handler returns, so
in real-world usage where signal handlers quickly return, it should
not be a problem. it's possible to solve this problem too by having
sigaction() wrap all signal handlers with a function that uses a
pthread_cleanup handler to catch cancellation, patch up the saved
context, and return into the cancellable function that will catch and
act upon the cancellation. however that would be a lot of complexity
for minimal if any benefit...

1 files changed, 4 insertions, 1 deletions

diff --git a/src/thread/sem_timedwait.c b/src/thread/sem_timedwait.c
index ad3bf075..11a01700 100644
--- a/src/thread/sem_timedwait.c
+++ b/src/thread/sem_timedwait.c
@@ -9,15 +9,18 @@ int sem_timedwait(sem_t *sem, const struct timespec *at)
 		if (a_fetch_add(sem->__val, -1) > 0) return 0;
 		val = a_fetch_add(sem->__val, 1)+1;
 		if (val==1) __wake(sem->__val, 1, 0);
-		CANCELPT_BEGIN;
 		if (at && at->tv_nsec >= 1000000000UL) {
 			errno = EINVAL;
 			return -1;
 		}
+		CANCELPT_BEGIN;
 		if (val <= 0 && __timedwait(sem->__val, val, CLOCK_REALTIME, at, 0) == ETIMEDOUT) {
 			errno = ETIMEDOUT;
+			CANCELPT_TRY;
+			CANCELPT_END;
 			return -1;
 		}
+		CANCELPT_TRY;
 		CANCELPT_END;
 	}
 }