about summary refs log tree commit diff
path: root/src/signal
Commit message (Collapse)AuthorAgeFilesLines
* fix external dummy_lock symbol inadvertently introduced in sigactionRich Felker2019-08-171-1/+1
| | | | | commit 9b14ad541068d4f7d0be9bcd1ff4c70090d868d3 introduced this namespace violation.
* setitimer, getitimer: decouple time_t from longRich Felker2019-07-294-0/+44
| | | | | | | | | | | | | | | | these functions have no new time64 syscall, so the existence of a time64 syscall cannot be used as the condition for the new code. instead, assume the syscall takes timevals as longs, which is true everywhere but x32, and interface with the kernel through long[4] objects. rather than adding new hacks to special-case x32 here, just add x32-specific source files since a trivial syscall wrapper suffices there. the new code paths added in this commit are statically unreachable on all current archs, but will become reachable when 32-bit archs get 64-bit time_t.
* sigtimedwait: add time64 syscall support, decouple 32-bit time_tRich Felker2019-07-281-4/+24
| | | | | | | | | | | | time64 syscall is used only if it's the only one defined for the arch, or if the requested timeout length does not fit in 32 bits. on current 32-bit archs where time_t is a 32-bit type, this makes it statically unreachable. on 64-bit archs, there are only superficial changes to the code after preprocessing. on current 32-bit archs, the timeout is passed via an intermediate copy to remove the assumption that time_t is a 32-bit type.
* fix sigaltstack to ignore ss_size with SS_DISABLE, per POSIXJames Y Knight2019-07-131-1/+1
|
* fix restrict violations in internal use of several functionsSamuel Holland2019-07-101-4/+4
| | | | | | | The old/new parameters to pthread_sigmask, sigprocmask, and setitimer are marked restrict, so passing the same address to both is prohibited. Modify callers of these functions to use a separate object for each argument.
* add riscv64 architecture supportRich Felker2019-06-142-0/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Author: Alex Suykov <alex.suykov@gmail.com> Author: Aric Belsito <lluixhi@gmail.com> Author: Drew DeVault <sir@cmpwn.com> Author: Michael Clark <mjc@sifive.com> Author: Michael Forney <mforney@mforney.org> Author: Stefan O'Rear <sorear2@gmail.com> This port has involved the work of many people over several years. I have tried to ensure that everyone with substantial contributions has been credited above; if any omissions are found they will be noted later in an update to the authors/contributors list in the COPYRIGHT file. The version committed here comes from the riscv/riscv-musl repo's commit 3fe7e2c75df78eef42dcdc352a55757729f451e2, with minor changes by me for issues found during final review: - a_ll/a_sc atomics are removed (according to the ISA spec, lr/sc are not safe to use in separate inline asm fragments) - a_cas[_p] is fixed to be a memory barrier - the call from the _start assembly into the C part of crt1/ldso is changed to allow for the possibility that the linker does not place them nearby each other. - DTP_OFFSET is defined correctly so that local-dynamic TLS works - reloc.h LDSO_ARCH logic is simplified and made explicit. - unused, non-functional crti/n asm files are removed. - an empty .sdata section is added to crt1 so that the __global_pointer reference is resolvable. - indentation style errors in some asm files are fixed.
* don't reject unknown/future flags in sigaltstack, allow SS_AUTODISARMRich Felker2019-03-051-1/+1
| | | | | | | | | | | | | | | | historically, and likely accidentally, sigaltstack was specified to fail with EINVAL if any flag bit other than SS_DISABLE was set. the resolution of Austin Group issue 1187 fixes this so that the requirement is only to fail for SS_ONSTACK (which cannot be set) or "invalid" flags. Linux fails on the kernel side for invalid flags, but historically accepts SS_ONSTACK as a no-op, so it needs to be rejected in userspace still. with this change, the Linux-specific SS_AUTODISARM, provided since commit 9680e1d03a794b0e0d5815c749478228ed40a36d but unusable due to rejection at runtime, is now usable.
* add __timedwait backend workaround for old kernels where futex EINTRsRich Felker2018-12-181-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | prior to linux 2.6.22, futex wait could fail with EINTR even for non-interrupting (SA_RESTART) signals. this was no problem provided the caller simply restarted the wait, but sem_[timed]wait is required by POSIX to return when interrupted by a signal. commit a113434cd68ce30642c4995b1caadcd084be6f09 introduced this behavior, and commit c0ed5a201b2bdb6d1896064bec0020c9973db0a1 reverted it based on a mistaken belief that it was not required. this belief stems from a bug in the specification: the description requires the function to return when interrupted, but the errors section marks EINTR as a "may fail" condition rather than a "shall fail" one. since there does seem to be significant value in the change made in commit c0ed5a201b2bdb6d1896064bec0020c9973db0a1, making it so that programs that call sem_wait without checking for EINTR don't silently make forward progress without obtaining the semaphore or treat it as a fatal error and abort, add a behind-the-scenes mechanism in the __timedwait backend to suppress EINTR in programs that have never installed interrupting signal handlers, and have sigaction track and report this state. this way the semaphore code is not cluttered by workarounds and can be updated (to be done in next commit) to reflect the high-level logic for conforming behavior. these changes are based loosely on a patch by Markus Wichmann, with the main changes being atomic update to flag object and moving the workaround from sem_timedwait to the __timedwait futex backend.
* split internal lock API out of libc.h, creating lock.hRich Felker2018-09-121-0/+1
| | | | | | | | | this further reduces the number of source files which need to include libc.h and thereby be potentially exposed to libc global state and internals. this will also facilitate further improvements like adding an inline fast-path, if we want to do so later.
* reduce spurious inclusion of libc.hRich Felker2018-09-124-4/+0
| | | | | | | | | | | | | | | | | | | | | libc.h was intended to be a header for access to global libc state and related interfaces, but ended up included all over the place because it was the way to get the weak_alias macro. most of the inclusions removed here are places where weak_alias was needed. a few were recently introduced for hidden. some go all the way back to when libc.h defined CANCELPT_BEGIN and _END, and all (wrongly implemented) cancellation points had to include it. remaining spurious users are mostly callers of the LOCK/UNLOCK macros and files that use the LFS64 macro to define the awful *64 aliases. in a few places, new inclusion of libc.h is added because several internal headers no longer implicitly include libc.h. declarations for __lockfile and __unlockfile are moved from libc.h to stdio_impl.h so that the latter does not need libc.h. putting them in libc.h made no sense at all, since the macros in stdio_impl.h are needed to use them correctly anyway.
* apply hidden visibility to sigreturn code fragmentsRich Felker2018-09-1214-2/+28
| | | | | | | these were overlooked in the declarations overhaul work because they are not properly declared, and the current framework even allows their declared types to vary by arch. at some point this should be cleaned up, but I'm not sure what the right way would be.
* overhaul internally-public declarations using wrapper headersRich Felker2018-09-121-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commits leading up to this one have moved the vast majority of libc-internal interface declarations to appropriate internal headers, allowing them to be type-checked and setting the stage to limit their visibility. the ones that have not yet been moved are mostly namespace-protected aliases for standard/public interfaces, which exist to facilitate implementing plain C functions in terms of POSIX functionality, or C or POSIX functionality in terms of extensions that are not standardized. some don't quite fit this description, but are "internally public" interfacs between subsystems of libc. rather than create a number of newly-named headers to declare these functions, and having to add explicit include directives for them to every source file where they're needed, I have introduced a method of wrapping the corresponding public headers. parallel to the public headers in $(srcdir)/include, we now have wrappers in $(srcdir)/src/include that come earlier in the include path order. they include the public header they're wrapping, then add declarations for namespace-protected versions of the same interfaces and any "internally public" interfaces for the subsystem they correspond to. along these lines, the wrapper for features.h is now responsible for the definition of the hidden, weak, and weak_alias macros. this means source files will no longer need to include any special headers to access these features. over time, it is my expectation that the scope of what is "internally public" will expand, reducing the number of source files which need to include *_impl.h and related headers down to those which are actually implementing the corresponding subsystems, not just using them.
* define and use internal macros for hidden visibility, weak refsRich Felker2018-09-052-3/+4
| | | | | | | | | this cleans up what had become widespread direct inline use of "GNU C" style attributes directly in the source, and lowers the barrier to increased use of hidden visibility, which will be useful to recovering some of the efficiency lost when the protected visibility hack was dropped in commit dc2f368e565c37728b0d620380b849c3a1ddd78f, especially on archs where the PLT ABI is costly.
* consistently use _NSIG/8 idiom for kernel sigset size in sigactionRich Felker2018-09-011-3/+3
| | | | | | | this code in sigaction was the only place where sizeof was being applied to the kernel sigaction's mask member to get the size argument to pass to the kernel. everywhere else, _NSIG/8 is used for this purpose.
* always terminate by SIGABRT when abort is calledRich Felker2018-09-011-4/+21
| | | | | | | | Linux makes this surprisingly difficult, but it can be done. the trick here is using the fact that we control the implementation of sigaction to prevent changing the disposition of SIGABRT to anything but SIG_DFL after abort has tried and failed to terminate the process simply by calling raise(SIGABRT).
* optimize raise not to make a syscall for getting tidRich Felker2018-09-011-3/+1
| | | | | assuming signals are blocked, which they are here, the tid in the thread structure is always valid and cannot change out from under us.
* prevent psignal/psiginfo from clobbering stderr orientation, errnoRich Felker2018-08-302-8/+21
| | | | | | | | | | | these functions are specified to write to stderr but not set its orientation, presumably so that they can be used in programs operating stderr in wide mode. also, they are not allowed to clobber errno on success. save and restore to meet the requirement. psiginfo is reduced to a think wrapper around psignal, since it already behaved the same. if we want to add more detailed siginfo printing at some point this will need refactoring.
* add m68k portRich Felker2018-06-191-0/+29
| | | | | | | | | | | | | three ABIs are supported: the default with 68881 80-bit fpu format and results returned in floating point registers, softfloat-only with the same format, and coldfire fpu with IEEE single/double only. only the first is tested at all, and only under qemu which has fpu emulation bugs. basic functionality smoke tests have been performed for the most common arch-specific breakage via libc-test and qemu user-level emulation. some sysvipc failures remain, but are shared with other big endian archs and will be fixed separately.
* optimize sigisemptysetRich Felker2018-05-011-2/+3
| | | | | | | | the static const zero set ended up getting put in bss instead of rodata, wasting writable memory, and the call to memcmp was size-inefficient. generally for nonstandard extension functions we try to avoid poking at any internals directly, but the way the zero set was setup was arguably already doing so.
* add public interface headers to implementation filesRich Felker2018-02-251-0/+2
| | | | | | | | | general policy is that all source files defining a public API or an ABI mechanism referenced by a public header should include the public header that declares the interface, so that the compiler or analysis tools can check the consistency of the declarations. Alexander Monakov pointed out a number of violations of this principle a few years back. fix them now.
* ppc64: fix setjmp/longjmp handling of TOC pointerBobby Bingham2017-08-111-7/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | The TOC pointer is constant within a single dso, but needs to be saved and restored around cross-dso calls. The PLT stub saves it to the caller's stack frame, and the linker adds code to the caller to restore it. With a local call, as within a single dso or with static linking, this doesn't happen and the TOC pointer is always in r2. Therefore, setjmp/longjmp need to save/restore the TOC pointer from/to different locations depending on whether the call to setjmp was a local or non-local call. It is always safe for longjmp to restore to both r2 and the caller's stack. If the call to setjmp was local, and only r2 matters and the stack location will be ignored, but is required by the ABI to be reserved for the TOC pointer. If the call was non-local, then only the stack location matters, and whatever is restored into r2 will be clobbered anyway when the caller reloads r2 from the stack. A little extra care is required for sigsetjmp, because it uses setjmp internally. After the second return from this setjmp call, r2 will contain the caller's TOC pointer instead of libc's TOC pointer. We need to save and restore the correct libc pointer before we can tail call to __sigsetjmp_tail.
* fix crashing sigsetjmp on s390xBobby Bingham2016-12-161-1/+1
| | | | | | the bz instruction that was wrongly used only admits a small immediate displacement and cannot be used with external symbols; apparently the linker fails to diagnose the overflow.
* work around gdb issues recognizing sigreturn trampoline on x86_64Rich Felker2016-11-122-8/+6
| | | | | | | | | | | | | | | | | | | | | | | | gdb can only backtrace/unwind across signal handlers if it recognizes the sa_restorer trampoline. for x86_64, gdb first attempts to determine the symbol name for the function in which the program counter resides and match it against "__restore_rt". if no name can be found (e.g. in the case of a stripped binary), the exact instruction sequence is matched instead. when matching the function name, however, gdb's unwind code wrongly considers the interval [sym,sym+size] rather than [sym,sym+size). thus, if __restore_rt begins immediately after another function, gdb wrongly identifies pc as lying within the previous adjacent function. this patch adds a nop before __restore_rt to preclude that possibility. it also removes the symbol name __restore and replaces it with a macro since the stability of whether gdb identifies the function as __restore_rt or __restore is not clear. for the no-symbols case, the instruction sequence is changed to use %rax rather than %eax to match what gdb expects. based on patch by Szabolcs Nagy, with extended description and corresponding x32 changes added.
* add s390x portBobby Bingham2016-11-112-0/+32
|
* add powerpc64 portBobby Bingham2016-05-082-0/+41
|
* add mips n32 port (ILP32 ABI for mips64)Rich Felker2016-04-182-0/+47
| | | | based on patch submitted by Jaydeep Patil, with minor changes.
* add mips64 portRich Felker2016-03-062-0/+47
| | | | | patch by Mahesh Bodapati and Jaydeep Patil of Imagination Technologies.
* fix crash when signal number 0 is passed to sigactionRich Felker2015-12-151-5/+1
| | | | | | this error case was overlooked in the old range checking logic. new check is moved out of __libc_sigaction to the public wrapper in order to unify the error path and reduce code size.
* eliminate use of SHARED macro to suppress visibility attributesRich Felker2015-11-111-2/+0
| | | | | | | | | | | | | | | | this is the first and simplest stage of removal of the SHARED macro, which will eventually allow libc.a and libc.so to be produced from the same object files. the original motivation for these #ifdefs which are now being removed was to allow building a static-only libc using a compiler that does not support visibility. however, SHARED was the wrong condition to test for this anyway; various assembly-language sources refer to hidden symbols and declare them with the .hidden directive, making it wrong to define the referenced symbols as non-hidden. if there is a need in the future to build libc using compilers that lack visibility, support could be moved to the build system or perhaps the __PIC__ macro could be checked instead of SHARED.
* explicitly assemble all arm asm sources as UALRich Felker2015-11-102-0/+3
| | | | | | | | these files are all accepted as legacy arm syntax when producing arm code, but legacy syntax cannot be used for producing thumb2 with access to the full ISA. even after switching to UAL, some asm source files contain instructions which are not valid in thumb mode, so these will need to be addressed separately.
* fix signal return for sh/fdpicRich Felker2015-09-232-4/+0
| | | | | | | | | | | | | | | | the restorer function pointer provided in the kernel sigaction structure is interpreted by the kernel as a raw code address, not a function descriptor. this commit moves the declarations of the __restore and __restore_rt symbols to ksigaction.h so that arch versions of the file can override them, and introduces a version for sh which declares them as objects rather than functions. an alternate solution would have been defining SA_RESTORER to 0 so that the functions are not used, but this both requires executable stack (since the sh kernel does not have a vdso page with permanent restorer functions) and crashes on qemu user-level emulation.
* remove unused (and invalid) C version of sigsetjmpRich Felker2015-09-091-17/+0
| | | | | | | | | originally, the comment in this code was correct and it would likely work if the compiler generated a tail call to setjmp. however, commit 583e55122e767b1586286a0d9c35e2a4027998ab redesigned sigsetjmp and siglongjmp such that the old C implementation (which was not intended to be used) is not even conceptually correct. remove it in the interest of avoiding confusion when porting to new archs.
* switch to using trap number 31 for syscalls on shRich Felker2015-06-161-2/+2
| | | | | | | | | | | | | | | | | | | nominally the low bits of the trap number on sh are the number of syscall arguments, but they have never been used by the kernel, and some code making syscalls does not even know the number of arguments and needs to pass an arbitrary high number anyway. sh3/sh4 traditionally used the trap range 16-31 for syscalls, but part of this range overlapped with hardware exceptions/interrupts on sh2 hardware, so an incompatible range 32-47 was chosen for sh2. using trap number 31 everywhere, since it's in the existing sh3/sh4 range and does not conflict with sh2 hardware, is a proposed unification of the kernel syscall convention that will allow binaries to be shared between sh2 and sh3/sh4. if this is not accepted into the kernel, we can refit the sh2 target with runtime selection mechanisms for the trap number, but doing so would be invasive and would entail non-trivial overhead.
* fix crash in x32 sigsetjmpRich Felker2015-05-021-0/+1
| | | | | | | | | | | the 64-bit push reads not only the 32-bit return address but also the first 32 signal mask bits. if any were nonzero, the return address obtained will be invalid. at some point storage of the return address should probably be moved to follow the saved mask so that there's plenty room and the same code can be used on x32 and regular x86_64, but for now I want a fix that does not risk breaking x86_64, and this simple re-zeroing works.
* fix sh jmp_buf size to match ABIRich Felker2015-04-271-2/+2
| | | | | | | | | | | | | | | | | while the sh port is still experimental and subject to ABI instability, this is not actually an application/libc boundary ABI change. it only affects third-party APIs where jmp_buf is used in a shared structure at the ABI boundary, because nothing anywhere near the end of the jmp_buf object (which includes the oversized sigset_t) is accessed by libc. both glibc and uclibc have 15-slot jmp_buf for sh. presumably the smaller version was used in musl because the slots for fpu status register and thread pointer register (gbr) were incorrect and must not be restored by longjmp, but the size should have been preserved, as it's generally treated as a libc-agnostic ABI property for the arch, and having extra slots free in case we ever need them for something is useful anyway.
* fix build regression in aarch64 sigsetjmpRich Felker2015-04-241-2/+2
| | | | | at least some assembler versions do not accept the register name lr. use the name x30 instead.
* remove invalid PLT calls from or1k asmRich Felker2015-04-191-3/+4
| | | | analogous to commit 646cb9a4a04e5ed78e2dd928bf9dc6e79202f609 for sh.
* remove possible-textrels from powerpc asmRich Felker2015-04-191-2/+3
| | | | | | | these are perfectly fine with ld-time symbol binding, but otherwise result in textrels. they cannot be replaced with @PLT jump targets because the PLT thunks require a GOT register to be setup, so use a hidden alias instead.
* remove invalid PLT calls from microblaze asmRich Felker2015-04-191-2/+3
| | | | analogous to commit 646cb9a4a04e5ed78e2dd928bf9dc6e79202f609 for sh.
* remove invalid PLT calls from sh asmRich Felker2015-04-191-2/+3
| | | | | | these are perfectly fine with ld-time symbol binding, but if the calls go through a PLT thunk, they are invalid because the caller does not setup a GOT register. use a hidden alias to bypass the issue.
* remove potentially PIC-incompatible relocations from x86_64 and x32 asmRich Felker2015-04-182-4/+4
| | | | analogous to commit 8ed66ecbcba1dd0f899f22b534aac92a282f42d5 for i386.
* remove the last of possible-textrels from i386 asmRich Felker2015-04-181-2/+3
| | | | | | | | | | | | none of these are actual textrels because of ld-time binding performed by -Bsymbolic-functions, but I'm changing them with the goal of making ld-time binding purely an optimization rather than relying on it for semantic purposes. in the case of memmove's call to memcpy, making it explicit that the memmove asm is assuming the forward-copying behavior of the memcpy asm is desirable anyway; in case memcpy is ever changed, the semantic mismatch would be apparent while editing memmcpy.s.
* redesign sigsetjmp so that signal mask is restored after longjmpRich Felker2015-04-1712-133/+177
| | | | | | | | | | | | | | | | | | | | | | | | | | | | the conventional way to implement sigsetjmp is to save the signal mask then tail-call to setjmp; siglongjmp then restores the signal mask and calls longjmp. the problem with this approach is that a signal already pending, or arriving between unmasking of signals and restoration of the saved stack pointer, will have its signal handler run on the stack that was active before siglongjmp was called. this can lead to unbounded stack usage when siglongjmp is used to leave a signal handler. in the new design, sigsetjmp saves its own return address inside the extended part of the sigjmp_buf (outside the __jmp_buf part used by setjmp) then calls setjmp to save a jmp_buf inside its own execution. it then tail-calls to __sigsetjmp_tail, which uses the return value of setjmp to determine whether to save the current signal mask or restore a previously-saved mask. as an added bonus, this design makes it so that siglongjmp and longjmp are identical. this is useful because the __longjmp_chk function we need to add for ABI-compatibility assumes siglongjmp and longjmp are the same, but for different reasons -- it was designed assuming either can access a flag just past the __jmp_buf indicating whether the signal masked was saved, and act on that flag. however, early versions of musl did not have space past the __jmp_buf for the non-sigjmp_buf version of jmp_buf, so our setjmp cannot store such a flag without risking clobbering memory on (very) old binaries.
* add aarch64 portSzabolcs Nagy2015-03-112-0/+27
| | | | | | | | | | This adds complete aarch64 target support including bigendian subarch. Some of the long double math functions are known to be broken otherwise interfaces should be fully functional, but at this point consider this port experimental. Initial work on this port was done by Sireesh Tripurari and Kevin Bortis.
* use tkill instead of tgkill in implementing raiseRich Felker2014-12-181-3/+2
| | | | | | | | | | this shaves off a useless syscall for getting the caller's pid and brings raise into alignment with other functions which were adapted to use tkill rather than tgkill. commit 83dc6eb087633abcf5608ad651d3b525ca2ec35e documents the rationale for this change, and in particular why the tgkill syscall is useless for its designed purpose of avoiding races.
* add or1k (OpenRISC 1000) architecture portStefan Kristiansson2014-07-181-0/+22
| | | | | | | | | | | | | | | With the exception of a fenv implementation, the port is fully featured. The port has been tested in or1ksim, the golden reference functional simulator for OpenRISC 1000. It passes all libc-test tests (except the math tests that requires a fenv implementation). The port assumes an or1k implementation that has support for atomic instructions (l.lwa/l.swa). Although it passes all the libc-test tests, the port is still in an experimental state, and has yet experienced very little 'real-world' use.
* add __sysv_signal abi-compat alias for the signal functionRich Felker2014-06-221-0/+1
| | | | | | | | | | it should be noted that the "real" __sysv_signal, which we do not implement, is semantically different from signal. references to __sysv_signal arise in code built against glibc under certain combinations of feature test macros, and are almost surely unintentional since the legacy sysv signal behavior has fundamental race conditions that cannot be worked around and which make it impossible to use safely.
* add __sigsetjmp ABI-compat alias for sigsetjmpRich Felker2014-04-029-1/+28
|
* always initialize thread pointer at program startRich Felker2014-03-241-4/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | this is the first step in an overhaul aimed at greatly simplifying and optimizing everything dealing with thread-local state. previously, the thread pointer was initialized lazily on first access, or at program startup if stack protector was in use, or at certain random places where inconsistent state could be reached if it were not initialized early. while believed to be fully correct, the logic was fragile and non-obvious. in the first phase of the thread pointer overhaul, support is retained (and in some cases improved) for systems/situation where loading the thread pointer fails, e.g. old kernels. some notes on specific changes: - the confusing use of libc.main_thread as an indicator that the thread pointer is initialized is eliminated in favor of an explicit has_thread_pointer predicate. - sigaction no longer needs to ensure that the thread pointer is initialized before installing a signal handler (this was needed to prevent a situation where the signal handler caused the thread pointer to be initialized and the subsequent sigreturn cleared it again) but it still needs to ensure that implementation-internal thread-related signals are not blocked. - pthread tsd initialization for the main thread is deferred in a new manner to minimize bloat in the static-linked __init_tp code. - pthread_setcancelstate no longer needs special handling for the situation before the thread pointer is initialized. it simply fails on systems that cannot support a thread pointer, which are non-conforming anyway. - pthread_cleanup_push/pop now check for missing thread pointer and nop themselves out in this case, so stdio no longer needs to avoid the cancellable path when the thread pointer is not available. a number of cases remain where certain interfaces may crash if the system does not support a thread pointer. at this point, these should be limited to pthread interfaces, and the number of such cases should be fewer than before.
* fix mips sigsetjmp asm to match fixed jmp_buf sizeRich Felker2014-03-181-1/+1
| | | | this was missed in the previous commit.