about summary refs log tree commit diff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* i386 vsyscall support (vdso-provided sysenter/syscall instruction based)Rich Felker2012-10-113-16/+62
| | | | | | this doubles the performance of the fastest syscalls on the atom I tested it on; improvement is reportedly much more dramatic on worst-case cpus. cannot be used for cancellable syscalls.
* ensure that buffer for decoding auxv at startup is initially zeroRich Felker2012-10-081-1/+1
|
* clean up and refactor program initializationRich Felker2012-10-076-34/+33
| | | | | | | | | | | | | | | the code in __libc_start_main is now responsible for parsing auxv, rather than duplicating the parsing all over the place. this should shave off a few cycles and some code size. __init_libc is left as an external-linkage function despite the fact that it could be static, to prevent it from being inlined and permanently wasting stack space when main is called. a few other minor changes are included, like eliminating per-thread ssp canaries (they were likely broken when combined with certain dlopen usages, and completely unnecessary) and some other unnecessary checks. since this code gets linked into every program, it should be as small and simple as possible.
* fix breakage due to initializing thread pointer when loading libsRich Felker2012-10-071-1/+1
| | | | | | at initial program load, all libraries must be loaded before the thread pointer can be setup, since the TP-relative addresses of all initial TLS objects must be constant.
* make new TLS setup block even implementation-internals signalsRich Felker2012-10-061-2/+1
| | | | | | | | | | this is needed to ensure async-cancel-safety, i.e. to make it safe to access TLS objects when async cancellation is enabled. otherwise, if cancellation were acter upon after the atomic fetch/add but before the thread saved the obtained memory, another access to the same TLS in the cancellation handler could end up performing the atomic fetch/add again, consuming more memory than is actually available and overflowing into other objects on the heap.
* don't crash if TLS library is loaded into process with no thread pointerRich Felker2012-10-061-0/+5
|
* fix buggy TLS size/alignment computations in static-linked TLSRich Felker2012-10-061-5/+22
|
* fix symbol acceptance/rejection rules for TLSRich Felker2012-10-061-8/+14
| | | | | | | | symbol value of 0 is not "undefined" for TLS; it's the address of the first symbol in the TLS segment. however, non-definition TLS references also have values of 0, so check the section. hopefully the new logic is more clear, too.
* TLS fixes, mainly alignment handlingRich Felker2012-10-061-39/+48
| | | | | | | | | | | | | compute offsets from the thread pointer statically when loading the library, rather than repeating the logic on each thread creation. not only is the latter less efficient at runtime; it also fails to provide solid guarantees that the offsets will remain the same when the initial alignment of memory is different. the new alignment handling is both more rigorous and simpler. the old code was also clobbering TLS bss with random image data in some cases due to using tls_size (size of TLS segment) instead of tls_len (length of the TLS data image).
* fix/improve shared library ctor/dtor handling, allow recursive dlopenRich Felker2012-10-051-7/+29
| | | | | | | | | | | | | | | | | | | | some libraries call dlopen from their constructors, resulting in recursive calls to dlopen. previously, this resulted in deadlock. I'm now unlocking the dlopen lock before running constructors (this is especially important since the lock also blocked pthread_create and was being held while application code runs!) and using a separate recursive mutex protecting the ctor/dtor state instead. in order to prevent the same ctor from being called more than once, a module is considered "constructed" just before the ctor runs. also, switch from using atexit to register each dtor to using a single atexit call to register the dynamic linker's dtor processing as just one handler. this is necessary because atexit performs allocation and may fail, but the library has already been loaded and cannot be backed-out at the time dtor registration is performed. this change also ensures that all dtors run after all atexit functions, rather than in mixed order.
* small dynamic linker module search fixRich Felker2012-10-051-1/+2
| | | | | | libraries loaded more than once by pathname should not get shortnames that would cause them to later be used to satisfy non-pathname load requests.
* support for TLS in dynamic-loaded (dlopen) modulesRich Felker2012-10-057-47/+115
| | | | | | | | | | | | | | | | | | | | | unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
* fix race condition in dlopenRich Felker2012-10-051-1/+3
| | | | | orig_tail was being saved before the lock was obtained, allowing dlopen failure to roll-back other dlopens that had succeeded.
* dynamic-linked TLS support for everything but dlopen'd libsRich Felker2012-10-041-38/+58
| | | | | | | | currently, only i386 is tested. x86_64 and arm should probably work. the necessary relocation types for mips and microblaze have not been added because I don't understand how they're supposed to work, and I'm not even sure if it's defined yet on microblaze. I may be able to reverse engineer the requirements out of gcc/binutils output.
* remove freeing of dynamic linker data when dlopen/dlsym are not usedRich Felker2012-10-041-11/+0
| | | | | | | | | | | | | | | | this was an optimization to save/recover a minimal amount of extra memory for use by malloc, that's becoming increasingly costly to keep around. freeing this data: 1. breaks debugging with gdb (it can't find library symbols) 2. breaks thread-local storage in shared libraries it would be possible to disable freeing when TLS is used, but in addition to the above breakages, tracking whether dlopen/dlsym is used adds a cost to every symbol lookup, possibly making program startup slower for large programs. combined with the complexity, it's not worth it. we already save/recover plenty of memory in the dynamic linker with reclaim_gaps.
* beginnings of full TLS support in shared librariesRich Felker2012-10-044-1/+19
| | | | | | this code will not work yet because the necessary relocations are not supported, and cannot be supported without some internal changes to how relocation processing works (coming soon).
* partial TLS support for dynamic-linked programsRich Felker2012-10-042-27/+77
| | | | | only TLS in the main program is supported so far; TLS defined in shared libraries will not work yet.
* TLS (GNU/C11 thread-local storage) support for static-linked programsRich Felker2012-10-046-14/+117
| | | | | | | | | | | | | the design for TLS in dynamic-linked programs is mostly complete too, but I have not yet implemented it. cost is nonzero but still low for programs which do not use TLS and/or do not use threads (a few hundred bytes of new code, plus dependency on memcpy). i believe it can be made smaller at some point by merging __init_tls and __init_security into __libc_start_main and avoiding duplicate auxv-parsing code. at the same time, I've also slightly changed the logic pthread_create uses to allocate guard pages to ensure that guard pages are not counted towards commit charge.
* add getopt reset supportRich Felker2012-09-302-2/+18
| | | | | | | based on proposed patches by Daniel Cegiełka, with minor changes: - use a weak symbol for optreset so it doesn't clash with namespace - also reset optpos (position in multi-option arg like -lR) - also make getopt_long support reset
* protect sem_open against cancellationRich Felker2012-09-301-13/+19
| | | | | also fix one minor bug: failure to free the early-reserved slot when the semaphore later found to already be mapped.
* overhaul sem_openRich Felker2012-09-302-108/+99
| | | | | | | | | | | this function was overly complicated and not even obviously correct. avoid using openat/linkat just like in shm_open, and instead expand pathname using code shared with shm_open. remove bogus (and dangerous, with priorities) use of spinlocks. this commit also heavily streamlines the code and ensures there are no failure cases that can happen after a new semaphore has been created in the filesystem, since that case is unreportable.
* clean up, bugfixes, and general improvement for shm_open/shm_unlinkRich Felker2012-09-302-30/+28
| | | | | | | 1. don't make non-cloexec file descriptors 2. cancellation safety (cleanup handlers were missing, now unneeded) 3. share name validation/mapping code between open/unlink functions 4. avoid wasteful/slow syscalls
* more close-on-exec fixes, mostly using new "e" flag to fopenRich Felker2012-09-298-9/+9
|
* add 'e' modifier (close-on-exec) to fopen and fdopenRich Felker2012-09-292-2/+5
| | | | | | this feature will be in the next version of POSIX, and can be used internally immediately. there are many internal uses of fopen where close-on-exec is needed to fix bugs.
* fix some more O_CLOEXEC/SOCK_CLOEXEC issuesRich Felker2012-09-294-7/+5
|
* move accept4, dup3, and pipe2 to non-linux-specific locationsRich Felker2012-09-293-0/+0
| | | | | these interfaces have been adopted by the Austin Group for inclusion in the next version of POSIX.
* emulate SOCK_CLOEXEC and SOCK_NONBLOCK for old (pre-2.6.27) kernelsRich Felker2012-09-292-3/+15
| | | | | | | | | | | | | | | also update syslog to use SOCK_CLOEXEC rather than separate fcntl step, to make it safe in multithreaded programs that run external programs. emulation is not atomic; it could be made atomic by holding a lock on forking during the operation, but this seems like overkill. my goal is not to achieve perfect behavior on old kernels (which have plenty of other imperfect behavior already) but to avoid catastrophic breakage in (1) syslog, which would give no output on old kernels with the change to use SOCK_CLOEXEC, and (2) programs built on a new kernel where configure scripts detected a working SOCK_CLOEXEC, which later get run on older kernels (they may otherwise fail to work completely).
* sem_open should make process-shared semaphoresRich Felker2012-09-291-1/+1
| | | | | this did not matter because we don't yet treat process-shared special. when private futex support is added, however, it will matter.
* use O_CLOEXEC to open semaphore files in sem_openRich Felker2012-09-291-2/+2
|
* fix some indention-with-spaces that crept inRich Felker2012-09-294-9/+9
|
* microblaze portRich Felker2012-09-2911-0/+188
| | | | | | based on initial work by rdp, with heavy modifications. some features including threads are untested because qemu app-level emulation seems to be broken and I do not have a proper system image for testing.
* fix arm clone syscall bug (no effect unless app uses clone)Rich Felker2012-09-271-2/+1
| | | | | the code to exit the new thread/process after the start function returns was mixed up in its syscall convention.
* optimize strchrnul/strcspn not to scan string twice on no-matchRich Felker2012-09-273-25/+29
| | | | | | | | | when strchr fails, and important piece of information already computed, the string length, is thrown away. have strchrnul (with namespace protection) be the underlying function so this information can be kept, and let strchr be a wrapper for it. this also allows strcspn to be considerably faster in the case where the match set has a single element that's not matched.
* slightly cleaner strlen, also seems to compile to better codeRich Felker2012-09-271-6/+4
| | | | | | | testing with gcc 4.6.3 on x86, -Os, the old version does a duplicate null byte check after the first loop. this is purely the compiler being stupid, but the old code was also stupid and unintuitive in how it expressed the check.
* fix dirname to handle input of form "foo/" correctlyRich Felker2012-09-261-6/+5
| | | | also optimized a bit.
* fix handling of EINTR during close()Rich Felker2012-09-241-1/+4
| | | | | | | | | | austin group interpretation for defect #529 (http://austingroupbugs.net/view.php?id=529) tightens the requirements on close such that, if it returns with EINTR, the file descriptor must not be closed. the linux kernel developers vehemently disagree with this, and will not change it. we catch and remap EINTR to EINPROGRESS, which the standard allows close() to return when the operation was not finished but the file descriptor has been closed.
* fix getaddrinfo to accept port 0 (zero)Rich Felker2012-09-221-2/+2
| | | | | | | | new behavior can be summarized as: inputs that parse completely as a decimal number are treated as one, and rejected only if the result is out of 16-bit range. inputs that do not parse as a decimal number (where strtoul leaves anything left over in the input) are searched in /etc/services.
* fix remaining IPC_64 issue (shmctl)Rich Felker2012-09-222-4/+2
| | | | also cleanup cruft related to the issue
* fix IPC_64 in msgctl tooRich Felker2012-09-222-6/+2
|
* fix broken semctl on systems that don't use IPC_64 flagRich Felker2012-09-221-2/+6
| | | | | not tested on mips and arm; they may still be broken. x86_64 should be ok now.
* LFS64 alias for prlimitRich Felker2012-09-211-0/+3
| | | | issue reported/requested by Justin Cormack
* add clock_adjtime, remap_file_pages, and syncfs syscall wrappersRich Felker2012-09-163-0/+25
| | | | patch by Justin Cormack, with slight modification
* add crypt_md5 password hashRich Felker2012-09-152-2/+285
| | | | contributed by nsz
* revert low rounds-count limits in crypt hashesRich Felker2012-09-154-4/+4
| | | | | | | | | | | | | | | it was determined in discussion that these kind of limits are not sufficient to protect single-threaded servers against denial of service attacks from maliciously large round counts. the time scales simply vary too much; many users will want login passwords with rounds counts on a scale that gives decisecond latency, while highly loaded webservers will need millisecond latency or shorter. still some limit is left in place; the idea is not to protect against attacks, but to avoid the runtime of a single call to crypt being, for all practical purposes, infinite, so that configuration errors can be caught and fixed without bringing down whole systems. these limits are very high, on the order of minute-long runtimes for modest systems.
* update mips cancellation-point syscall asm with 7-arg and r25 fixesRich Felker2012-09-151-2/+5
| | | | | these fixes were already made to the normal syscall asm but not the cancellation point version.
* workaround gcc got-register-reload performance problems in mallocRich Felker2012-09-141-4/+8
| | | | | | | with this patch, the malloc in libc.so built with -Os is nearly the same speed as the one built with -O3. thus it solves the performance regression that resulted from removing the forced -O3 when building libc.so; now libc.so can be both small and fast.
* use vfork if possible in posix_spawnRich Felker2012-09-141-1/+3
| | | | | | vfork is implemented as the fork syscall (with no atfork handlers run) on archs where it is not available, so this change does not introduce any change in behavior or regression for such archs.
* asm for memmove on i386 and x86_64Rich Felker2012-09-102-0/+36
| | | | | | | for the sake of simplicity, I've only used rep movsb rather than breaking up the copy for using rep movsd/q. on all modern cpus, this seems to be fine, but if there are performance problems, there might be a need to go back and add support for rep movsd/q.
* fix another ppoll issue (missing sigset_t size argument)Rich Felker2012-09-101-1/+1
|
* reenable word-at-at-time copying in memmoveRich Felker2012-09-101-4/+27
| | | | | | | | | before restrict was added, memove called memcpy for forward copies and used a byte-at-a-time loop for reverse copies. this was changed to avoid invoking UB now that memcpy has an undefined copying order, making memmove considerably slower. performance is still rather bad, so I'll be adding asm soon.