redesign sigsetjmp so that signal mask is restored after longjmp

the conventional way to implement sigsetjmp is to save the signal mask
then tail-call to setjmp; siglongjmp then restores the signal mask and
calls longjmp. the problem with this approach is that a signal already
pending, or arriving between unmasking of signals and restoration of
the saved stack pointer, will have its signal handler run on the stack
that was active before siglongjmp was called. this can lead to
unbounded stack usage when siglongjmp is used to leave a signal
handler.

in the new design, sigsetjmp saves its own return address inside the
extended part of the sigjmp_buf (outside the __jmp_buf part used by
setjmp) then calls setjmp to save a jmp_buf inside its own execution.
it then tail-calls to __sigsetjmp_tail, which uses the return value of
setjmp to determine whether to save the current signal mask or restore
a previously-saved mask.

as an added bonus, this design makes it so that siglongjmp and longjmp
are identical. this is useful because the __longjmp_chk function we
need to add for ABI-compatibility assumes siglongjmp and longjmp are
the same, but for different reasons -- it was designed assuming either
can access a flag just past the __jmp_buf indicating whether the
signal masked was saved, and act on that flag. however, early versions
of musl did not have space past the __jmp_buf for the non-sigjmp_buf
version of jmp_buf, so our setjmp cannot store such a flag without
risking clobbering memory on (very) old binaries.

1 files changed, 16 insertions, 9 deletions

diff --git a/src/signal/x86_64/sigsetjmp.s b/src/signal/x86_64/sigsetjmp.s
index 17436f02..d354d680 100644
--- a/src/signal/x86_64/sigsetjmp.s
+++ b/src/signal/x86_64/sigsetjmp.s
@@ -1,17 +1,24 @@
-/* Copyright 2011-2012 Nicholas J. Kain, licensed under standard MIT license */
 .global sigsetjmp
 .global __sigsetjmp
 .type sigsetjmp,@function
 .type __sigsetjmp,@function
 sigsetjmp:
 __sigsetjmp:
-	andl %esi,%esi
-	movq %rsi,64(%rdi)
+	test %esi,%esi
 	jz 1f
-	pushq %rdi
-	leaq 72(%rdi),%rdx
-	xorl %esi,%esi
-	movl $2,%edi
-	call sigprocmask
-	popq %rdi
+
+	popq 64(%rdi)
+	mov %rbx,72+8(%rdi)
+	mov %rdi,%rbx
+
+	call setjmp
+
+	pushq 64(%rbx)
+	mov %rbx,%rdi
+	mov %eax,%esi
+	mov 72+8(%rbx),%rbx
+
+.hidden __sigsetjmp_tail
+	jmp __sigsetjmp_tail
+
 1:	jmp setjmp