about summary refs log tree commit diff
path: root/src/internal
Commit message (Collapse)AuthorAgeFilesLines
* fix broken shgetc limiter logic (wasn't working)Rich Felker2012-04-162-2/+5
|
* floatscan: fix incorrect count of leading nonzero digitsRich Felker2012-04-161-1/+1
| | | | | | | this off-by-one error was causing values with just one digit past the decimal point to be treated by the integer case. in many cases it would yield the correct result, but if expressions are evaluated in excess precision, double rounding may occur.
* use fast version of the int reading code for the high-order digits tooRich Felker2012-04-131-3/+13
| | | | | this increases code size slightly, but it's considerably faster, especially for power-of-2 bases.
* use macros instead of inline functions in shgetc.hRich Felker2012-04-131-20/+4
| | | | | | at -Os optimization level, gcc refuses to inline these functions even though the inlined code would roughly the same size as the function call, and much faster. the easy solution is to make them into macros.
* fix spurious overflows in strtoull with small basesRich Felker2012-04-131-7/+3
| | | | | | | whenever the base was small enough that more than one digit could still fit after UINTMAX_MAX/36-1 was reached, only the first would be allowed; subsequent digits would trigger spurious overflow, making it impossible to read the largest values in low bases.
* remove magic numbers from floatscanRich Felker2012-04-121-5/+5
|
* optimize more integer cases in floatscan; comment the whole procedureRich Felker2012-04-121-8/+27
|
* revert invalid optimization in floatscanRich Felker2012-04-111-2/+2
|
* fix stupid typo in floatscan that caused excess rounding of some valuesRich Felker2012-04-111-1/+1
|
* optimize floatscan downscaler to skip results that won't be neededRich Felker2012-04-111-2/+3
| | | | | | | | | | | | | | when upscaling, even the very last digit is needed in cases where the input is exact; no digits can be discarded. but when downscaling, any digits less significant than the mantissa bits are destined for the great bitbucket; the only influence they can have is their presence (being nonzero). thus, we simply throw them away early. the result is nearly a 4x performance improvement for processing huge values. the particular threshold LD_B1B_DIG+3 is not chosen sharply; it's simply a "safe" distance past the significant bits. it would be nice to replace it with a sharp bound, but i suspect performance will be comparable (within a few percent) anyway.
* simplify/debloat radix point alignment code in floatscanRich Felker2012-04-111-9/+4
| | | | | | | | | | | | now that this is the first operation, it can rely on the circular buffer contents not being wrapped when it begins. we limit the number of digits read slightly in the initial parsing loops too so that this code does not have to consider the case where it might cause the circular buffer to wrap; this is perfectly fine because KMAX is chosen as a power of two for circular-buffer purposes and is much larger than it otherwise needs to be, anyway. these changes should not affect performance at all.
* optimize floatscan: avoid excessive upscalingRich Felker2012-04-111-27/+27
| | | | | | | | | | upscaling by even one step too much creates 3-29 extra iterations for the next loop. this is still suboptimal since it always goes by 2^29 rather than using a smaller upscale factor when nearing the target, but performance on common, small-magnitude, few-digit values has already more than doubled with this change. more optimizations on the way...
* fix incorrect initial count in shgetc when data is already bufferedRich Felker2012-04-111-1/+1
|
* fix bug parsing lone zero followed by junk, and hex float over-readingRich Felker2012-04-111-6/+5
|
* fix float scanning of certain values ending in zerosRich Felker2012-04-101-1/+3
| | | | | | | for example, "1000000000" was being read as "1" due to this loop exiting early. it's necessary to actually update z and zero the entries so that the subsequent rounding code does not get confused; before i did that, spurious inexact exceptions were being raised.
* fix potential overflow in exponent readingRich Felker2012-04-101-1/+1
| | | | | | | note that there's no need for a precise cutoff, because exponents this large will always result in overflow or underflow (it's impossible to read enough digits to compensate for the exponent magnitude; even at a few nanoseconds per digit it would take hundreds of years).
* set errno properly when parsing floating pointRich Felker2012-04-101-4/+21
|
* add "scan helper getc" and rework strtod, etc. to use itRich Felker2012-04-105-73/+111
| | | | | | | | | | | | | | | | | | the immediate benefit is a significant debloating of the float parsing code by moving the responsibility for keeping track of the number of characters read to a different module. by linking shgetc with the stdio buffer logic, counting logic is defered to buffer refill time, keeping the calls to shgetc fast and light. in the future, shgetc will also be useful for integrating the new float code with scanf, which needs to not only count the characters consumed, but also limit the number of characters read based on field width specifiers. shgetc may also become a useful tool for simplifying the integer parsing code.
* new floating point parser/converterRich Felker2012-04-102-0/+446
| | | | | | | | | | | | | | | | | this version is intended to be fully conformant to the ISO C, POSIX, and IEEE standards for conversion of decimal/hex floating point strings to float, double, and long double (ld64 or ld80 only at present) values. in particular, all results are intended to be rounded correctly according to the current rounding mode. further, this implementation aims to set the floating point underflow, overflow, and inexact flags to reflect the conversion performed. a moderate amount of testing has been performed (by nsz and myself) prior to integration of the code in musl, but it still may have bugs. so far, only strto(d|ld|f) use the new code. scanf integration will be done as a separate commit, and i will add implementations of the wide character functions later.
* add creal/cimag macros in complex.h (and use them in the functions defs)Rich Felker2012-03-221-8/+0
|
* don't inline __rem_pio2l so the code size is smallernsz2012-03-191-0/+1
|
* fix loads of missing const in new libm, and some global vars (?!) in powlRich Felker2012-03-181-2/+2
|
* fix namespace issues for lgamma, etc.Rich Felker2012-03-161-0/+2
| | | | standard functions cannot depend on nonstandard symbols
* first commit of the new libm!Rich Felker2012-03-132-0/+323
| | | | | | | | | | | | | | | | thanks to the hard work of Szabolcs Nagy (nsz), identifying the best (from correctness and license standpoint) implementations from freebsd and openbsd and cleaning them up! musl should now fully support c99 float and long double math functions, and has near-complete complex math support. tgmath should also work (fully on gcc-compatible compilers, and mostly on any c99 compiler). based largely on commit 0376d44a890fea261506f1fc63833e7a686dca19 from nsz's libm git repo, with some additions (dummy versions of a few missing long double complex functions, etc.) by me. various cleanups still need to be made, including re-adding (if they're correct) some asm functions that were dropped.
* fix obscure bug in strtoull reading the highest 16 possible valuesRich Felker2012-03-021-1/+1
|
* new attempt at working around the gcc 3 visibility bugRich Felker2012-02-242-0/+7
| | | | | since gcc is failing to generate the necessary ".hidden" directive in the output asm, generate it explicitly with an __asm__ statement...
* remove useless attribute visibility from definitionsRich Felker2012-02-241-1/+1
| | | | | | this was a failed attempt at working around the gcc 3 visibility bug affecting x86_64. subsequent patch will address it with an ugly but working hack.
* cleanup and work around visibility bug in gcc 3 that affects x86_64Rich Felker2012-02-232-6/+11
| | | | | | | | | | | | | | in gcc 3, the visibility attribute must be placed on both the declaration and on the definition. if it's omitted from the definition, the compiler fails to emit the ".hidden" directive in the assembly, and the linker will either generate textrels (if supported, such as on i386) or refuse to link (on targets where certain types of textrels are forbidden or impossible without further assumptions about memory layout, such as on x86_64). this patch also unifies the decision about when to use visibility into libc.h and makes the visibility in the utf-8 state machine tables based on libc.h rather than a duplicate test.
* synchronize cond var destruction with exiting waitsRich Felker2011-10-021-0/+1
|
* improve pshared barriersRich Felker2011-09-281-1/+1
| | | | | | | | | | | | | | eliminate the sequence number field and instead use the counter as the futex because of the way the lock is held, sequence numbers are completely useless, and this frees up a field in the barrier structure to be used as a waiter count for the count futex, which lets us avoid some syscalls in the best case. as of now, self-synchronized destruction and unmapping should be fully safe. before any thread can return from the barrier, all threads in the barrier have obtained the vm lock, and each holds a shared lock on the barrier. the barrier memory is not inspected after the shared lock count reaches 0, nor after the vm lock is released.
* process-shared barrier support, based on discussion with bdonlanRich Felker2011-09-271-3/+5
| | | | | | | | | | | | | this implementation is rather heavy-weight, but it's the first solution i've found that's actually correct. all waiters actually wait twice at the barrier so that they can synchronize exit, and they hold a "vm lock" that prevents changes to virtual memory mappings (and blocks pthread_barrier_destroy) until all waiters are finished inspecting the barrier. thus, it is safe for any thread to destroy and/or unmap the barrier's memory as soon as pthread_barrier_wait returns, without further synchronization.
* fix lost signals in cond varsRich Felker2011-09-261-0/+1
| | | | | | | | | | | due to moving waiters from the cond var to the mutex in bcast, these waiters upon wakeup would steal slots in the count from newer waiters that had not yet been signaled, preventing the signal function from taking any action. to solve the problem, we simply use two separate waiter counts, and so that the original "total" waiters count is undisturbed by broadcast and still available for signal.
* cleanup various minor issues reported by nszRich Felker2011-09-261-3/+3
| | | | | | | | | the changes to syscall_ret are mostly no-ops in the generated code, just cleanup of type issues and removal of some implementation-defined behavior. the one exception is the change in the comparison value, which is fixed so that 0xf...f000 (which in principle could be a valid return value for mmap, although probably never in reality) is not treated as an error return.
* redo cond vars again, use sequence numbersRich Felker2011-09-261-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | testing revealed that the old implementation, while correct, was giving way too many spurious wakeups due to races changing the value of the condition futex. in a test program with 5 threads receiving broadcast signals, the number of returns from pthread_cond_wait was roughly 3 times what it should have been (2 spurious wakeups for every legitimate wakeup). moreover, the magnitude of this effect seems to grow with the number of threads. the old implementation may also have had some nasty race conditions with reuse of the cond var with a new mutex. the new implementation is based on incrementing a sequence number with each signal event. this sequence number has nothing to do with the number of threads intended to be woken; it's only used to provide a value for the futex wait to avoid deadlock. in theory there is a danger of race conditions due to the value wrapping around after 2^32 signals. it would be nice to eliminate that, if there's a way. testing showed no spurious wakeups (though they are of course possible) with the new implementation, as well as slightly improved performance.
* new futex-requeue-based pthread_cond_broadcast implementationRich Felker2011-09-251-3/+6
| | | | | | this avoids the "stampede effect" where pthread_cond_broadcast would result in all waiters waking up simultaneously, only to immediately contend for the mutex and go back to sleep.
* fix deadlock in condition wait whenever there are multiple waitersRich Felker2011-09-221-0/+1
| | | | | it's amazing none of the conformance tests i've run even bothered to check whether something so basic works...
* initial commit of the arm portRich Felker2011-09-181-0/+15
| | | | | | | | | | | | | this port assumes eabi calling conventions, eabi linux syscall convention, and presence of the kernel helpers at 0xffff0f?0 needed for threads support. otherwise it makes very few assumptions, and the code should work even on armv4 without thumb support, as well as on systems with thumb interworking. the bits headers declare this a little endian system, but as far as i can tell the code should work equally well on big endian. some small details are probably broken; so far, testing has been limited to qemu/aboriginal linux.
* overhaul clone syscall wrappingRich Felker2011-09-181-2/+1
| | | | | | | | | | | | | | | | | | | | several things are changed. first, i have removed the old __uniclone function signature and replaced it with the "standard" linux __clone/clone signature. this was necessary to expose clone to applications anyway, and it makes it easier to port __clone to new archs, since it's now testable independently of pthread_create. secondly, i have removed all references to the ugly ldt descriptor structure (i386 only) from the c code and pthread structure. in places where it is needed, it is now created on the stack just when it's needed, in assembly code. thus, the i386 __clone function takes the desired thread pointer as its argument, rather than an ldt descriptor pointer, just like on all other sane archs. this should not affect applications since there is really no way an application can use clone with threads/tls in a way that doesn't horribly conflict with and clobber the underlying implementation's use. applications are expected to use clone only for creating actual processes, possibly with new namespace features and whatnot.
* security hardening: ensure suid programs have valid stdin/out/errRich Felker2011-08-231-2/+4
| | | | | | | | | | | this behavior (opening fds 0-2 for a suid program) is explicitly allowed (but not required) by POSIX to protect badly-written suid programs from clobbering files they later open. this commit does add some cost in startup code, but the availability of auxv and the security flag will be useful elsewhere in the future. in particular auxv is needed for static-linked vdso support, which is still waiting to be committed (sorry nik!)
* pthread and synccall cleanup, new __synccall_wait opRich Felker2011-08-122-0/+2
| | | | | | | | | fix up clone signature to match the actual behavior. the new __syncall_wait function allows a __synccall callback to wait for other threads to continue without returning, so that it can resume action after the caller finishes. this interface could be made significantly more general/powerful with minimal effort, but i'll wait to do that until it's actually useful for something.
* simplify multi-threaded errno, eliminate useless function pointerRich Felker2011-08-061-2/+1
|
* use weak aliases rather than function pointers to simplify some codeRich Felker2011-08-061-2/+0
|
* overhaul rwlocks to address several issuesRich Felker2011-08-031-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | like mutexes and semaphores, rwlocks suffered from a race condition where the unlock operation could access the lock memory after another thread successfully obtained the lock (and possibly destroyed or unmapped the object). this has been fixed in the same way it was fixed for other lock types. in addition, the previous implementation favored writers over readers. in the absence of other considerations, that is the best behavior for rwlocks, and posix explicitly allows it. however posix also requires read locks to be recursive. if writers are favored, any attempt to obtain a read lock while a writer is waiting for the lock will fail, causing "recursive" read locks to deadlock. this can be avoided by keeping track of which threads already hold read locks, but doing so requires unbounded memory usage, and there must be a fallback case that favors readers in case memory allocation failed. and all of this must be synchronized. the cost, complexity, and risk of errors in getting it right is too great, so we simply favor readers. tracking of the owner of write locks has been removed, as it was not useful for anything. it could allow deadlock detection, but it's not clear to me that returning EDEADLK (which a buggy program is likely to ignore) is better than deadlocking; at least the latter behavior prevents further data corruption. a correct program cannot invoke this situation anyway. the reader count and write lock state, as well as the "last minute" waiter flag have all been combined into a single atomic lock. this means all state transitions for the lock are atomic compare-and-swap operations. this makes establishing correctness much easier and may improve performance. finally, some code duplication has been cleaned up. more is called for, especially the standard __timedwait idiom repeated in all locks.
* unify and overhaul timed futex waitsRich Felker2011-08-022-2/+4
| | | | | | | | | | | | | | new features: - FUTEX_WAIT_BITSET op will be used for timed waits if available. this saves a call to clock_gettime. - error checking for the timespec struct is now inside __timedwait so it doesn't need to be duplicated everywhere. cond_timedwait still needs to duplicate it to avoid unlocking the mutex, though. - pushing and popping the cancellation handler is delegated to __timedwait, and cancellable/non-cancellable waits are unified.
* add proper fuxed-based locking for stdioRich Felker2011-07-303-6/+8
| | | | | | | | | | | | | | | | | | | | | | previously, stdio used spinlocks, which would be unacceptable if we ever add support for thread priorities, and which yielded pathologically bad performance if an application attempted to use flockfile on a key file as a major/primary locking mechanism. i had held off on making this change for fear that it would hurt performance in the non-threaded case, but actually support for recursive locking had already inflicted that cost. by having the internal locking functions store a flag indicating whether they need to perform unlocking, rather than using the actual recursive lock counter, i was able to combine the conditionals at unlock time, eliminating any additional cost, and also avoid a nasty corner case where a huge number of calls to ftrylockfile could cause deadlock later at the point of internal locking. this commit also fixes some issues with usage of pthread_self conflicting with __attribute__((const)) which resulted in crashes with some compiler versions/optimizations, mainly in flockfile prior to pthread_create.
* new attempt at making set*id() safe and robustRich Felker2011-07-292-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | changing credentials in a multi-threaded program is extremely difficult on linux because it requires synchronizing the change between all threads, which have their own thread-local credentials on the kernel side. this is further complicated by the fact that changing the real uid can fail due to exceeding RLIMIT_NPROC, making it possible that the syscall will succeed in some threads but fail in others. the old __rsyscall approach being replaced was robust in that it would report failure if any one thread failed, but in this case, the program would be left in an inconsistent state where individual threads might have different uid. (this was not as bad as glibc, which would sometimes even fail to report the failure entirely!) the new approach being committed refuses to change real user id when it cannot temporarily set the rlimit to infinity. this is completely POSIX conformant since POSIX does not require an implementation to allow real-user-id changes for non-privileged processes whatsoever. still, setting the real uid can fail due to memory allocation in the kernel, but this can only happen if there is not already a cached object for the target user. thus, we forcibly serialize the syscalls attempts, and fail the entire operation on the first failure. this *should* lead to an all-or-nothing success/failure result, but it's still fragile and highly dependent on kernel developers not breaking things worse than they're already broken. ideally linux will eventually add a CLONE_USERCRED flag that would give POSIX conformant credential changes without any hacks from userspace, and all of this code would become redundant and could be removed ~10 years down the line when everyone has abandoned the old broken kernels. i'm not holding my breath...
* comment non-obvious de bruijn sequence code in int parserRich Felker2011-07-251-0/+2
|
* fix various bugs in new integer parser frameworkRich Felker2011-07-141-4/+7
| | | | | | | | | | 1. my interpretation of subject sequence definition was wrong. adjust parser to conform to the standard. 2. some code for handling tail overflow case was missing (forgot to finish writing it). 3. typo (= instead of ==) caused ERANGE to wrongly behave like EINVAL
* new restartable integer parsing framework.Rich Felker2011-07-142-0/+116
| | | | | | | | | this fixes a number of bugs in integer parsing due to lazy haphazard wrapping, as well as some misinterpretations of the standard. the new parser is able to work character-at-a-time or on whole strings, making it easy to support the wide functions without unbounded space for conversion. it will also be possible to update scanf to use the new parser.
* restore use of .type in asm, but use modern @function (vs %function)Rich Felker2011-06-142-0/+2
| | | | | | | | this seems to be necessary to make the linker accept the functions in a shared library (perhaps to generate PLT entries?) strictly speaking libc-internal asm should not need it. i might clean that up later.