about summary refs log tree commit diff
path: root/src/ldso
Commit message (Collapse)AuthorAgeFilesLines
* add general fdpic support in dynamic linker and arch support for shRich Felker2015-09-222-11/+188
| | | | | | | | | | | | | | | | | | at this point not all functionality is complete. the dynamic linker itself, and main app if it is also loaded by the kernel, take advantage of fdpic and do not need constant displacement between segments, but additional libraries loaded by the dynamic linker follow normal ELF semantics for mapping still. this fully works, but does not admit shared text on nommu. in terms of actual functional correctness, dlsym's results are presently incorrect for function symbols, RTLD_NEXT fails to identify the caller correctly, and dladdr fails almost entirely. with the dynamic linker entry point working, support for static pie is automatically included, but linking the main application as ET_DYN (pie) probably does not make sense for fdpic anyway. ET_EXEC is equally relocatable but more efficient at representing relocations.
* factor symbol counting out of dladdr as its own functionRich Felker2015-09-211-19/+20
| | | | | | | | | | | | | the fdpic code will need to count symbols, and it may be useful elsewhere in the future too. counting is trivial as long as sysv hash is present, but for gnu-hash-only libraries it's complex. the behavior of the count is changed slightly: we now include symbols that are not accessible by the gnu hash table in the count. this may make dladdr slightly slower. if this is a problem, dladdr can subtract out the part that should not be accessible. unlike in the old code, subtracting this out is easy even in the fast path where sysv hash is available too.
* simplify dlstart code by using integer type for base addressRich Felker2015-09-211-8/+7
|
* refactor some more dynamic linker load address computationsRich Felker2015-09-171-7/+7
| | | | these were just missed in the previous commits.
* remove some useless casts in dynamic linkerRich Felker2015-09-171-2/+2
|
* further refactoring of dynamic linker load address computationsRich Felker2015-09-171-2/+2
| | | | | | | these are in do_relocs. the first one was omitted in commit 301335a80b85f12c018e4acf1a2c28615e119f8d because it slightly changes code (using dso->base rather than cached local var base) and would have prevented easy verification. the other was an oversight.
* begin refactoring load address computations in dynamic linkerRich Felker2015-09-171-19/+22
| | | | | | | | | | | | | for ordinary ELF with fixed segment displacements, load address computation is simply adding the base load address. but for FDPIC, each segment has its own load address, and virtual addresses need to be adjusted according to the segment they fall in. abstracting this computation is the first step to making the dynamic linker ready for FDPIC. for this first commit, a macro is used rather than a function in order to facilitate correctness checking. I have verified that the generated code does not change on my i386 build.
* remove old dlstart stage-2 symbolic lookup code; add new genericRich Felker2015-09-171-14/+8
| | | | | | | | | | | | | | this new generic version of the stage-2 function lookup should work for any arch where static data is accessible via got-relative or pc-relative addressing, using approximately the technique described in the log message for commit 2907afb8dbd4c1d34825c3c9bd2b41564baca210. since all the mips-like archs that need got slots fo access static data have already transitioned to the new stage chaining scheme, the old dynamic symbol lookup code is now removed. aarch64, arm, and sh have not yet transitioned; with this commit, they are now using the new generic code.
* introduce new symbol-lookup-free rcrt1/dlstart stage chainingRich Felker2015-09-171-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | previously, the call into stage 2 was made by looking up the symbol name "__dls2" (which was chosen short to be easy to look up) from the dynamic symbol table. this was no problem for the dynamic linker, since it always exports all its symbols. in the case of the static pie entry point, however, the dynamic symbol table does not contain the necessary symbol unless -rdynamic/-E was used when linking. this linking requirement is a major obstacle both to practical use of static-pie as a nommu binary format (since it greatly enlarges the file) and to upstream toolchain support for static-pie (adding -E to default linking specs is not reasonable). this patch replaces the runtime symbolic lookup with a link-time lookup via an inline asm fragment, which reloc.h is responsible for providing. in this initial commit, the asm is provided only for i386, and the old lookup code is left in place as a fallback for archs that have not yet transitioned. modifying crt_arch.h to pass the stage-2 function pointer as an argument was considered as an alternative, but such an approach would not be compatible with fdpic, where it's impossible to compute function pointers without already having performed relocations. it was also deemed desirable to keep crt_arch.h as simple/minimal as possible. in principle, archs with pc-relative or got-relative addressing of static variables could instead load the stage-2 function pointer from a static volatile object. that does not work for fdpic, and is not safe against reordering on mips-like archs that use got slots even for static functions, but it's a valid on i386 and many others, and could provide a reasonable default implementation in the future.
* dynlink.c: pass gnu-hash table pointer to gnu_lookupAlexander Monakov2015-06-281-13/+11
| | | | | | | | | | The callers need to check the value of the pointer anyway, so make them pass the pointer to gnu_lookup instead of reloading it there. Reorder gnu_lookup arguments so that always-used ones are listed first. GCC can choose a calling convention with arguments in registers (e.g. up to 3 arguments in eax, ecx, edx on x86), but cannot reorder the arguments for static functions.
* dynlink.c: slim down gnu_lookupAlexander Monakov2015-06-281-9/+5
| | | | | Do not reference dso->syms and dso->strings until point of use. Check 'h1 == (h2|1)', the simplest condition, before the others.
* dynlink.c: use bloom filter in gnu hash lookupAlexander Monakov2015-06-281-3/+22
| | | | | | | | Introduce gnu_lookup_filtered and use it to speed up symbol lookups in find_sym (do_dlsym is left as is, based on an expectation that frequently dlsym queries will use a dlopen handle rather than RTLD_NEXT or RTLD_DEFAULT, and will not need to look at more than one DSO).
* dynlink.c: use a faster expression in gnu_hashAlexander Monakov2015-06-271-1/+1
| | | | | With -Os, GCC uses a multiply rather than a shift and addition for 'h*33'. Use a more efficient expression explicitely.
* fix local-dynamic model TLS on mips and powerpcRich Felker2015-06-251-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | the TLS ABI spec for mips, powerpc, and some other (presently unsupported) RISC archs has the return value of __tls_get_addr offset by +0x8000 and the result of DTPOFF relocations offset by -0x8000. I had previously assumed this part of the ABI was actually just an implementation detail, since the adjustments cancel out. however, when the local dynamic model is used for accessing TLS that's known to be in the same DSO, either of the following may happen: 1. the -0x8000 offset may already be applied to the argument structure passed to __tls_get_addr at ld time, without any opportunity for runtime relocations. 2. __tls_get_addr may be used with a zero offset argument to obtain a base address for the module's TLS, to which the caller then applies immediate offsets for individual objects accessed using the local dynamic model. since the immediate offsets have the -0x8000 adjustment applied to them, the base address they use needs to include the +0x8000 offset. it would be possible, but more complex, to store the pointers in the dtv[] array with the +0x8000 offset pre-applied, to avoid the runtime cost of adding 0x8000 on each call to __tls_get_addr. this change could be made later if measurements show that it would help.
* make dynamic linker work around MAP_FAILED mmap failure on nommu kernelsRich Felker2015-06-231-2/+24
| | | | | | previously, loading of additional libraries beyond libc/ldso did not work on nommu kernels, nor did loading programs via invocation of the dynamic linker as a command.
* ignore ENOSYS error from mprotect in pthread_create and dynamic linkerRich Felker2015-06-171-2/+4
| | | | | this error simply indicated a system without memory protection (NOMMU) and should not cause failure in the caller.
* fix regression in pre-v7 arm on kernels with kuser helper removedRich Felker2015-06-071-17/+14
| | | | | | | | | | | | | | | the arm atomics/TLS runtime selection code is called from __set_thread_area and depends on having libc.auxv and __hwcap available. commit 71f099cb7db821c51d8f39dfac622c61e54d794c moved the first call to __set_thread_area to the top of dynamic linking stage 3, before this data is made available, causing the runtime detection code to always see __hwcap as zero and thereby select the atomics/TLS implementations based on kuser helper. upcoming work on superh will use similar runtime detection. ideally this early-init code should be cleanly refactored and shared between the dynamic linker and static-linked startup.
* fix dynamic linker regression processing R_*_NONE type relocationsRich Felker2015-06-041-0/+1
| | | | | | | | | | | | | | | commit f3ddd173806fd5c60b3f034528ca24542aecc5b9 inadvertently removed the early check for "none" type relocations, causing the address dso->base+0 to be dereferenced to obtain an addend. shared libraries, (including libc.so) and PIE executables were unaffected, since their base addresses are the actual address of their mappings and are readable. non-PIE main executables, however, have a base address of 0 because their load addresses are absolute and not offset at load time. in practice none-type relocations do not arise with toolchains that are in use except on mips, and on mips it's moderately rare for a non-PIE executable to have a relocation table, since the mips-specific got processing serves in its place for most purposes.
* reprocess all libc/ldso symbolic relocations in dynamic linking stage 3Rich Felker2015-05-251-11/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit f3ddd173806fd5c60b3f034528ca24542aecc5b9 introduced early relocations and subsequent reprocessing as part of the dynamic linker bootstrap overhaul, to allow use of arbitrary libc functions before the main application and libraries are loaded, but only reprocessed GOT/PLT relocation types. commit c093e2e8201524db0d638920e76bcb6b1d925f3a added reprocessing of non-GOT/PLT relocations to fix an actual regression that was observed on powerpc, but only for RELA format tables with out-of-line addends. REL table (inline addends at the relocation address) reprocessing is trickier because the first relocation pass clobbers the addends. this patch extends symbolic relocation reprocessing for libc/ldso to support all relocation types, whether REL or RELA format tables are used. it is believed not to alter behavior on any existing archs for the current dynamic linker and libc code. the motivations for this change are consistency and future-proofing. it ensures that behavior does not differ depending on whether REL or RELA tables are used, which could lead to undetected arch-specific bugs. it also ensures that, if in the future code depending on additional relocation types is added to libc.so, either at the source level or as part of the compiler runtime that gets pulled in (for example, soft-float with TLS for fenv), the new code will work properly. the implementation concept is simple: stage 2 of the dynamic linker counts the number of symbolic relocations in the libc/ldso REL table and allocates a VLA to save their addends into; stage 3 then uses the saved addends in place of the inline ones which were clobbered. for stack safety, a hard limit (currently 4k) is imposed on the number of such addends; this should be a couple orders of magnitude larger than the actual need. this number is not a runtime variable that could break fail-safety; it is constant for a given libc.so build.
* move call to dynamic linker stage-3 into stage-2 functionRich Felker2015-05-252-11/+8
| | | | | | | | | this move eliminates a duplicate "by-hand" symbol lookup loop from the stage-1 code and replaces it with a call to find_sym, which can be used once we're in stage 2. it reduces the size of the stage 1 code, which is helpful because stage 1 will become the crt start file for static-PIE executables, and it will allow stage 3 to access stage 2's automatic storage, which will be important in an upcoming commit.
* simplify/shrink relocation processing in dynamic linker stage 1Rich Felker2015-05-251-24/+16
| | | | | | | | | the outer-loop approach made sense when we were also processing DT_JMPREL, which might be in REL or RELA form, to avoid major code duplication. commit 09db855b35709aa627d7055c57a98e1e471920ab removed processing of DT_JMPREL, and in the remaining two tables, the format (REL or RELA) is known by the name of the table. simply writing two versions of the loop results in smaller and simpler code.
* remove processing of DT_JMPREL from dynamic linker stage 1 bootstrapRich Felker2015-05-251-1/+0
| | | | | | | the DT_JMPREL relocation table necessarily consists entirely of JMP_SLOT (REL_PLT in internal nomenclature) relocations, which are symbolic; they cannot be resolved in stage 1, so there is no point in processing them.
* reprocess libc/ldso RELA relocations in stage 3 of dynamic linkingRich Felker2015-05-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | this fixes a regression on powerpc that was introduced in commit f3ddd173806fd5c60b3f034528ca24542aecc5b9. global data accesses on powerpc seem to be using a translation-unit-local GOT filled via R_PPC_ADDR32 relocations rather than R_PPC_GLOB_DAT. being a non-GOT relocation type, these were not reprocessed after adding the main application and its libraries to the chain, causing libc code not to see copy relocations in the main program, and therefore to use the pre-copy-relocation addresses for global data objects (like environ). the motivation for the dynamic linker only reprocessing GOT/PLT relocation types in stage 3 is that these types always have a zero addend, making them safe to process again even if the storage for the addend has been clobbered. other relocation types which can be used for address constants in initialized data objects may have non-zero addends which will be clobbered during the first pass of relocation processing if they're stored inline (REL form) rather than out-of-line (RELA form). powerpc generally uses only RELA, so this patch is sufficient to fix the regression in practice, but is not fully general, and would not suffice if an alternate toolchain generated REL for powerpc.
* remove always-true conditional in dynamic linker TLSDESC processingRich Felker2015-04-211-1/+1
| | | | | | the allocating path which can fail is for dynamic TLS, which can only occur at runtime, and the check for runtime was already made in the outer conditional.
* remove redundant code in do_dlsym functionRich Felker2015-04-211-1/+1
| | | | | | | commit 637dd2d383cc1f63bf02a732f03786857b22c7bd introduced the checks for RTLD_DEFAULT and RTLD_NEXT here, claiming they fixed a regression, but the above conditional block clearly already covered these cases, and removing the checks produces no difference in the generated code.
* make dlerror state and message thread-local and dynamically-allocatedRich Felker2015-04-181-32/+61
| | | | | | | | | this fixes truncation of error messages containing long pathnames or symbol names. the dlerror state was previously required by POSIX to be global. the resolution of bug 97 relaxed the requirements to allow thread-safe implementations of dlerror with thread-local state and message buffer.
* apply hidden visibility to tlsdesc accessor functionsRich Felker2015-04-175-0/+10
| | | | | | | these functions are never called directly; only their addresses are used, so PLT indirections should never happen unless a broken application tries to redefine them, but it's still best to make them hidden.
* comment fixes in aarch64 tlsdesc asmSzabolcs Nagy2015-04-171-4/+4
|
* ensure debugger hook for dynamic linker does not point to a PLT slotRich Felker2015-04-171-2/+4
| | | | | this change is made in preparation to support linking without -Bsymbolic-functions.
* fix PLT call offset in sh dlsym asmRich Felker2015-04-171-3/+3
| | | | | | | | | | | the braf instruction's destination register is an offset from the address of the braf instruction plus 4 (or equivalently, the address of the next instruction after the delay slot). the code for dlsym was incorrectly computing the offset to pass using the address of the delay slot itself. in other places, a label was placed after the delay slot, but I find this confusing. putting the label on the branch instruction itself, and manually adding 4, makes it more clear which branch the offset in the constant pool goes with.
* fix sh build regressions in asmRich Felker2015-04-171-1/+1
| | | | | even hidden functions need @PLT symbol references; otherwise an absolute address is produced instead of a PC-relative one.
* use hidden __tls_get_new for tls/tlsdesc lookup fallback casesRich Felker2015-04-143-4/+10
| | | | | | | | | | | | | | | previously, the dynamic tlsdesc lookup functions and the i386 special-ABI ___tls_get_addr (3 underscores) function called __tls_get_addr when the slot they wanted was not already setup; __tls_get_addr would then in turn also see that it's not setup and call __tls_get_new. calling __tls_get_new directly is both more efficient and avoids the issue of calling a non-hidden (public API/ABI) function from asm. for the special i386 function, a weak reference to __tls_get_new is used since this function is not defined when static linking (the code path that needs it is unreachable in static-linked programs).
* use hidden visibility for call from dlsym to internal __dlsymRich Felker2015-04-1411-3/+14
|
* fix inconsistent visibility for internal __tls_get_new functionRich Felker2015-04-141-0/+1
| | | | | | at the point of call it was declared hidden, but the definition was not hidden. for some toolchains this inconsistency produced textrels without ld-time binding.
* make _dlstart_c function use hidden visibilityRich Felker2015-04-141-0/+1
| | | | | otherwise the call/jump from the crt_arch.h asm may not resolve correctly without -Bsymbolic-functions.
* remove initializers for decoded aux/dyn arrays in dynamic linkerRich Felker2015-04-131-5/+5
| | | | | | | | the zero initialization is redundant since decode_vec does its own clearing, and it increases the risk that buggy compilers will generate calls to memset. as long as symbols are bound at ld time, such a call will not break anything, but it may be desirable to turn off ld-time binding in the future.
* remove remnants of support for running in no-thread-pointer modeRich Felker2015-04-131-10/+2
| | | | | | | | | | | | | since 1.1.0, musl has nominally required a thread pointer to be setup. most of the remaining code that was checking for its availability was doing so for the sake of being usable by the dynamic linker. as of commit 71f099cb7db821c51d8f39dfac622c61e54d794c, this is no longer necessary; the thread pointer is now valid before any libc code (outside of dynamic linker bootstrap functions) runs. this commit essentially concludes "phase 3" of the "transition path for removing lazy init of thread pointer" project that began during the 1.1.0 release cycle.
* move thread pointer setup to beginning of dynamic linker stage 3Rich Felker2015-04-131-8/+23
| | | | | | | | | | | | this allows the dynamic linker itself to run with a valid thread pointer, which is a prerequisite for stack protector on archs where the ssp canary is stored in TLS. it will also allow us to remove some remaining runtime checks for whether the thread pointer is valid. as long as the application and its libraries do not require additional size or alignment, this early thread pointer will be kept and reused at runtime. otherwise, a new static TLS block is allocated after library loading has finished and the thread pointer is switched over.
* stabilize dynamic linker's layout of static TLSRich Felker2015-04-131-9/+6
| | | | | | | | | | | | | | previously, the layout of the static TLS block was perturbed by the size of the dtv; dtv size increasing from 0 to 1 perturbed both TLS arch types, and the TLS-above-TP type's layout was perturbed by the specific number of dtv slots (libraries with TLS). this behavior made it virtually impossible to setup a tentative thread pointer address before loading libraries and keep it unchanged as long as the libraries' TLS size/alignment requirements fit. the new code fixes the location of the dtv and pthread structure at opposite ends of the static TLS block so that they will not move unless size or alignment changes.
* dynamic linker bootstrap overhaulRich Felker2015-04-1313-442/+281
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | this overhaul further reduces the amount of arch-specific code needed by the dynamic linker and removes a number of assumptions, including: - that symbolic function references inside libc are bound at link time via the linker option -Bsymbolic-functions. - that libc functions used by the dynamic linker do not require access to data symbols. - that static/internal function calls and data accesses can be made without performing any relocations, or that arch-specific startup code handled any such relocations needed. removing these assumptions paves the way for allowing libc.so itself to be built with stack protector (among other things), and is achieved by a three-stage bootstrap process: 1. relative relocations are processed with a flat function. 2. symbolic relocations are processed with no external calls/data. 3. main program and dependency libs are processed with a fully-functional libc/ldso. reduction in arch-specific code is achived through the following: - crt_arch.h, used for generating crt1.o, now provides the entry point for the dynamic linker too. - asm is no longer responsible for skipping the beginning of argv[] when ldso is invoked as a command. - the functionality previously provided by __reloc_self for heavily GOT-dependent RISC archs is now the arch-agnostic stage-1. - arch-specific relocation type codes are mapped directly as macros rather than via an inline translation function/switch statement.
* fix rpath string memory leak on failed dlopenRich Felker2015-04-041-0/+2
| | | | | | | | when dlopen fails, all partially-loaded libraries need to be unmapped and freed. any of these libraries using an rpath with $ORIGIN expansion may have an allocated string for the expanded rpath; previously, this string was not freed when freeing the library data structures.
* halt dynamic linker library search on errors resolving $ORIGIN in rpathRich Felker2015-04-031-8/+18
| | | | | | | | | | | | | | | this change hardens the dynamic linker against the possibility of loading the wrong library due to inability to expand $ORIGIN in rpath. hard failures such as excessively long paths or absence of /proc (when resolving /proc/self/exe for the main executable's origin) do not stop the path search, but memory allocation failures and any other potentially transient failures do. to implement this change, the meaning of the return value of fixup_rpath function is changed. returning zero no longer indicates that the dso's rpath string pointer is non-null; instead, the caller needs to check. a return value of -1 indicates a failure that should stop further path search.
* harden dynamic linker library path searchRich Felker2015-04-011-5/+16
| | | | | | | | | | | | transient errors during the path search should not allow the search to continue and possibly open the wrong file. this patch eliminates most conditions where that could happen, but there is still a possibility that $ORIGIN-based rpath processing will have an allocation failure, causing the search to skip such a path. fixing this is left as a separate task. a small bug where overly-long path components caused an infinite loop rather than being skipped/ignored is also fixed.
* add aarch64 portSzabolcs Nagy2015-03-113-0/+115
| | | | | | | | | | This adds complete aarch64 target support including bigendian subarch. Some of the long double math functions are known to be broken otherwise interfaces should be fully functional, but at this point consider this port experimental. Initial work on this port was done by Sireesh Tripurari and Kevin Bortis.
* copy the dtv pointer to the end of the pthread struct for TLS_ABOVE_TP archsSzabolcs Nagy2015-03-111-3/+3
| | | | | | | | | | | | | | | | | | | | There are two main abi variants for thread local storage layout: (1) TLS is above the thread pointer at a fixed offset and the pthread struct is below that. So the end of the struct is at known offset. (2) the thread pointer points to the pthread struct and TLS starts below it. So the start of the struct is at known (zero) offset. Assembly code for the dynamic TLSDESC callback needs to access the dynamic thread vector (dtv) pointer which is currently at the front of the pthread struct. So in case of (1) the asm code needs to hard code the offset from the end of the struct which can easily break if the struct changes. This commit adds a copy of the dtv at the end of the struct. New members must not be added after dtv_copy, only before it. The size of the struct is increased a bit, but there is opportunity for size optimizations.
* fix over-alignment of TLS, insufficient builtin TLS on 64-bit archsRich Felker2015-03-061-2/+8
| | | | | | | | | | | | | a conservative estimate of 4*sizeof(size_t) was used as the minimum alignment for thread-local storage, despite the only requirements being alignment suitable for struct pthread and void* (which struct pthread already contains). additional alignment required by the application or libraries is encoded in their headers and is already applied. over-alignment prevented the builtin_tls array from ever being used in dynamic-linked programs on 64-bit archs, thereby requiring allocation at startup even in programs with no TLS of their own.
* make all objects used with atomic operations volatileRich Felker2015-03-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the memory model we use internally for atomics permits plain loads of values which may be subject to concurrent modification without requiring that a special load function be used. since a compiler is free to make transformations that alter the number of loads or the way in which loads are performed, the compiler is theoretically free to break this usage. the most obvious concern is with atomic cas constructs: something of the form tmp=*p;a_cas(p,tmp,f(tmp)); could be transformed to a_cas(p,*p,f(*p)); where the latter is intended to show multiple loads of *p whose resulting values might fail to be equal; this would break the atomicity of the whole operation. but even more fundamental breakage is possible. with the changes being made now, objects that may be modified by atomics are modeled as volatile, and the atomic operations performed on them by other threads are modeled as asynchronous stores by hardware which happens to be acting on the request of another thread. such modeling of course does not itself address memory synchronization between cores/cpus, but that aspect was already handled. this all seems less than ideal, but it's the best we can do without mandating a C11 compiler and using the C11 model for atomics. in the case of pthread_once_t, the ABI type of the underlying object is not volatile-qualified. so we are assuming that accessing the object through a volatile-qualified lvalue via casts yields volatile access semantics. the language of the C standard is somewhat unclear on this matter, but this is an assumption the linux kernel also makes, and seems to be the correct interpretation of the standard.
* adapt dynamic linker for new binutils versions that omit DT_RPATHRich Felker2014-11-231-0/+2
| | | | | | | | the new DT_RUNPATH semantics for search order are always used, and since binutils had always set both DT_RPATH and DT_RUNPATH when the latter was used, processing only DT_RPATH worked fine. however, recent binutils has stopped generating DT_RPATH when DT_RUNPATH is used, which broke support for this feature completely.
* save auxv pointer into libc struct early in dynamic linker startupRich Felker2014-11-191-0/+1
| | | | | this allows most code to assume it has already been saved, and is a prerequisite for upcoming changes for arm atomic/tls operations.
* add dlerror message for static-linked dlsym failureRich Felker2014-08-081-0/+2
|