summary refs log tree commit diff
path: root/malloc/arena.c
Commit message (Collapse)AuthorAgeFilesLines
* Update copyright dates with scripts/update-copyrightsPaul Eggert2022-01-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | I used these shell commands: ../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright (cd ../glibc && git commit -am"[this commit message]") and then ignored the output, which consisted lines saying "FOO: warning: copyright statement not found" for each of 7061 files FOO. I then removed trailing white space from math/tgmath.h, support/tst-support-open-dev-null-range.c, and sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following obscure pre-commit check failure diagnostics from Savannah. I don't know why I run into these diagnostics whereas others evidently do not. remote: *** 912-#endif remote: *** 913: remote: *** 914- remote: *** error: lines with trailing whitespace found ... remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
* malloc: Enable huge page support on main arenaAdhemerval Zanella2021-12-151-0/+4
| | | | | | | | | | | | This patch adds support huge page support on main arena allocation, enable with tunable glibc.malloc.hugetlb=2. The patch essentially disable the __glibc_morecore() sbrk() call (similar when memory tag does when sbrk() call does not support it) and fallback to default page size if the memory allocation fails. Checked on x86_64-linux-gnu. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Add Huge Page support to arenasAdhemerval Zanella2021-12-151-42/+92
| | | | | | | | | | | | | | | | | | | | | | | It is enabled as default for glibc.malloc.hugetlb set to 2 or higher. It also uses a non configurable minimum value and maximum value, currently set respectively to 1 and 4 selected huge page size. The arena allocation with huge pages does not use MAP_NORESERVE. As indicate by kernel internal documentation [1], the flag might trigger a SIGBUS on soft page faults if at memory access there is no left pages in the pool. On systems without a reserved huge pages pool, is just stress the mmap(MAP_HUGETLB) allocation failure. To improve test coverage it is required to create a pool with some allocated pages. Checked on x86_64-linux-gnu with no reserved pages, 10 reserved pages (which trigger mmap(MAP_HUGETBL) failures) and with 256 reserved pages (which does not trigger mmap(MAP_HUGETLB) failures). [1] https://www.kernel.org/doc/html/v4.18/vm/hugetlbfs_reserv.html#resv-map-modifications Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Add Huge Page support for mmapAdhemerval Zanella2021-12-151-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | With the morecore hook removed, there is not easy way to provide huge pages support on with glibc allocator without resorting to transparent huge pages. And some users and programs do prefer to use the huge pages directly instead of THP for multiple reasons: no splitting, re-merging by the VM, no TLB shootdowns for running processes, fast allocation from the reserve pool, no competition with the rest of the processes unlike THP, no swapping all, etc. This patch extends the 'glibc.malloc.hugetlb' tunable: the value '2' means to use huge pages directly with the system default size, while a positive value means and specific page size that is matched against the supported ones by the system. Currently only memory allocated on sysmalloc() is handled, the arenas still uses the default system page size. To test is a new rule is added tests-malloc-hugetlb2, which run the addes tests with the required GLIBC_TUNABLE setting. On systems without a reserved huge pages pool, is just stress the mmap(MAP_HUGETLB) allocation failure. To improve test coverage it is required to create a pool with some allocated pages. Checked on x86_64-linux-gnu. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Add madvise support for Transparent Huge PagesAdhemerval Zanella2021-12-151-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | Linux Transparent Huge Pages (THP) current supports three different states: 'never', 'madvise', and 'always'. The 'never' is self-explanatory and 'always' will enable THP for all anonymous pages. However, 'madvise' is still the default for some system and for such case THP will be only used if the memory range is explicity advertise by the program through a madvise(MADV_HUGEPAGE) call. To enable it a new tunable is provided, 'glibc.malloc.hugetlb', where setting to a value diffent than 0 enables the madvise call. This patch issues the madvise(MADV_HUGEPAGE) call after a successful mmap() call at sysmalloc() with sizes larger than the default huge page size. The madvise() call is disable is system does not support THP or if it has the mode set to "never" and on Linux only support one page size for THP, even if the architecture supports multiple sizes. To test is a new rule is added tests-malloc-hugetlb1, which run the addes tests with the required GLIBC_TUNABLE setting. Checked on x86_64-linux-gnu. Reviewed-by: DJ Delorie <dj@redhat.com>
* misc: Add __get_nprocs_schedAdhemerval Zanella2021-09-271-1/+1
| | | | | | | | | | | This is an internal function meant to return the number of avaliable processor where the process can scheduled, different than the __get_nprocs which returns a the system available online CPU. The Linux implementation currently only calls __get_nprocs(), which in tuns calls sched_getaffinity. Reviewed-by: Florian Weimer <fweimer@redhat.com>
* Remove "Contributed by" linesSiddhesh Poyarekar2021-09-031-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | We stopped adding "Contributed by" or similar lines in sources in 2012 in favour of git logs and keeping the Contributors section of the glibc manual up to date. Removing these lines makes the license header a bit more consistent across files and also removes the possibility of error in attribution when license blocks or files are copied across since the contributed-by lines don't actually reflect reality in those cases. Move all "Contributed by" and similar lines (Written by, Test by, etc.) into a new file CONTRIBUTED-BY to retain record of these contributions. These contributors are also mentioned in manual/contrib.texi, so we just maintain this additional record as a courtesy to the earlier developers. The following scripts were used to filter a list of files to edit in place and to clean up the CONTRIBUTED-BY file respectively. These were not added to the glibc sources because they're not expected to be of any use in future given that this is a one time task: https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dc https://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02 Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* Fix build and tests with --disable-tunablesSiddhesh Poyarekar2021-07-231-5/+0
| | | | | | | | | | | | | Remove unused code and declare __libc_mallopt when !IS_IN (libc) to allow the debug hook to build with --disable-tunables. Also, run tst-ifunc-isa-2* tests only when tunables are enabled since the result depends on it. Tested on x86_64. Reported-by: Matheus Castanho <msc@linux.ibm.com> Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* glibc.malloc.check: Wean away from malloc hooksSiddhesh Poyarekar2021-07-221-12/+5
| | | | | | | | | | | | | | | | | | The malloc-check debugging feature is tightly integrated into glibc malloc, so thanks to an idea from Florian Weimer, much of the malloc implementation has been moved into libc_malloc_debug.so to support malloc-check. Due to this, glibc malloc and malloc-check can no longer work together; they use altogether different (but identical) structures for heap management. This should not make a difference though since the malloc check hook is not disabled anywhere. malloc_set_state does, but it does so early enough that it shouldn't cause any problems. The malloc check tunable is now in the debug DSO and has no effect when the DSO is not preloaded. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com>
* Simplify __malloc_initializedSiddhesh Poyarekar2021-07-221-6/+6
| | | | | | | | | Now that mcheck no longer needs to check __malloc_initialized (and no other third party hook can since the symbol is not exported), make the variable boolean and static so that it is used strictly within malloc. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com>
* Move malloc hooks into a compat DSOSiddhesh Poyarekar2021-07-221-7/+0
| | | | | | | | | | | | | | | | | | | | | | | | Remove all malloc hook uses from core malloc functions and move it into a new library libc_malloc_debug.so. With this, the hooks now no longer have any effect on the core library. libc_malloc_debug.so is a malloc interposer that needs to be preloaded to get hooks functionality back so that the debugging features that depend on the hooks, i.e. malloc-check, mcheck and mtrace work again. Without the preloaded DSO these debugging features will be nops. These features will be ported away from hooks in subsequent patches. Similarly, legacy applications that need hooks functionality need to preload libc_malloc_debug.so. The symbols exported by libc_malloc_debug.so are maintained at exactly the same version as libc.so. Finally, static binaries will no longer be able to use malloc debugging features since they cannot preload the debugging DSO. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com>
* Remove __morecore and __default_morecoreSiddhesh Poyarekar2021-07-221-10/+2
| | | | | | | | | | Make the __morecore and __default_morecore symbols compat-only and remove their declarations from the API. Also, include morecore.c directly into malloc.c; this should ideally get merged into malloc in a future cleanup. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Tested-by: Carlos O'Donell <carlos@redhat.com>
* Harden tcache double-free checkSiddhesh Poyarekar2021-07-081-0/+8
| | | | | | | | | | | | | | | | | | | | The tcache allocator layer uses the tcache pointer as a key to identify a block that may be freed twice. Since this is in the application data area, an attacker exploiting a use-after-free could potentially get access to the entire tcache structure through this key. A detailed write-up was provided by Awarau here: https://awaraucom.wordpress.com/2020/07/19/house-of-io-remastered/ Replace this static pointer use for key checking with one that is generated at malloc initialization. The first attempt is through getrandom with a fallback to random_bits(), which is a simple pseudo-random number generator based on the clock. The fallback ought to be sufficient since the goal of the randomness is only to make the key arbitrary enough that it is very unlikely to collide with user data. Co-authored-by: Eyal Itkin <eyalit@checkpoint.com>
* malloc: Make tunable callback functions staticH.J. Lu2021-05-071-2/+2
| | | | | Since malloc tunable callback functions are only used within the same file, we should make them static.
* malloc: Use branches instead of mtag_granule_maskSzabolcs Nagy2021-03-261-1/+0
| | | | | | | | | | | | | The branches may be better optimized since mtag_enabled is widely used. Granule size larger than a chunk header is not supported since then we cannot have both the chunk header and user area granule aligned. To fix that for targets with large granule, the chunk layout has to change. So code that attempted to handle the granule mask generally was changed. This simplified CHUNK_AVAILABLE_SIZE and the logic in malloc_usable_size. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Use global flag instead of function pointer dispatch for mtagSzabolcs Nagy2021-03-261-32/+1
| | | | | | | | | | | | | | | | | | A flag check can be faster than function pointers because of how branch prediction and speculation works and it can also remove a layer of indirection when there is a mismatch between the malloc internal tag_* api and __libc_mtag_* target hooks. Memory tagging wrapper functions are moved to malloc.c from arena.c and the logic now checks mmap_enabled. The definition of tag_new_usable is moved after chunk related definitions. This refactoring also allows using mtag_enabled checks instead of USE_MTAG ifdefs when memory tagging support only changes code logic when memory tagging is enabled at runtime. Note: an "if (false)" code block is optimized away even at -O0 by gcc. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Refactor TAG_ macros to avoid indirectionSzabolcs Nagy2021-03-261-8/+8
| | | | | | | | | | | This does not change behaviour, just removes one layer of indirection in the internal memory tagging logic. Use tag_ and mtag_ prefixes instead of __tag_ and __mtag_ since these are all symbols with internal linkage, private to malloc.c, so there is no user namespace pollution issue. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Simplify __mtag_tag_new_usableSzabolcs Nagy2021-03-261-5/+0
| | | | | | | | | The chunk cannot be a dumped one here. The only non-obvious cases are free and realloc which may be called on a dumped area chunk, but in both cases it can be verified that tagging is already avoided for dumped area chunks. Reviewed-by: DJ Delorie <dj@redhat.com>
* Update copyright dates with scripts/update-copyrightsPaul Eggert2021-01-021-1/+1
| | | | | | | | | | | | | | | | I used these shell commands: ../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright (cd ../glibc && git commit -am"[this commit message]") and then ignored the output, which consisted lines saying "FOO: warning: copyright statement not found" for each of 6694 files FOO. I then removed trailing white space from benchtests/bench-pthread-locks.c and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this diagnostic from Savannah: remote: *** pre-commit check failed ... remote: *** error: lines with trailing whitespace found remote: error: hook declined to update refs/heads/master
* malloc: Basic support for memory tagging in the malloc() familyRichard Earnshaw2020-12-211-3/+56
| | | | | | | | | | | | | | | | | This patch adds the basic support for memory tagging. Various flavours are supported, particularly being able to turn on tagged memory at run-time: this allows the same code to be used on systems where memory tagging support is not present without neededing a separate build of glibc. Also, depending on whether the kernel supports it, the code will use mmap for the default arena if morecore does not, or cannot support tagged memory (on AArch64 it is not available). All the hooks use function pointers to allow this to work without needing ifuncs. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Use __libc_initial to detect an inner libcFlorian Weimer2020-12-161-8/+5
| | | | | | | | | | | | | The secondary/non-primary/inner libc (loaded via dlmopen, LD_AUDIT, static dlopen) must not use sbrk to allocate member because that would interfere with allocations in the outer libc. On Linux, this does not matter because sbrk itself was changed to fail in secondary libcs. _dl_addr occasionally shows up in profiles, but had to be used before because __libc_multiple_libs was unreliable. So this change achieves a slight reduction in startup time. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* malloc debug: fix compile error when enable macro MALLOC_DEBUG > 1liqingqing2020-10-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | malloc debug: fix compile error when enable macro MALLOC_DEBUG > 1. this is because commit e9c4fe93b3855239752819303ca377dff0ed0553 has change the struct malloc_chunk's member "size" to "mchunk_size". the reproduction is like that: setp1: modify related Makefile. vim ../glibc/malloc/Makefile CPPFLAGS-malloc.o += -DMALLOC_DEBUG=2 step2: ../configure --prefix=/usr make -j32 this will cause the compile error: /home/liqingqing/glibc_upstream/buildglibc/malloc/malloc.o In file included from malloc.c:1899:0: arena.c: In function 'dump_heap': arena.c:422:58: error: 'struct malloc_chunk' has no member named 'size' fprintf (stderr, "chunk %p size %10lx", p, (long) p->size); ^~ arena.c:428:17: error: 'struct malloc_chunk' has no member named 'size' else if (p->size == (0 | PREV_INUSE)) Reviewed-by: DJ Delorie <dj@redhat.com>
* Update copyright dates with scripts/update-copyrights.Joseph Myers2020-01-011-1/+1
|
* Prefer https to http for gnu.org and fsf.org URLsPaul Eggert2019-09-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also, change sources.redhat.com to sourceware.org. This patch was automatically generated by running the following shell script, which uses GNU sed, and which avoids modifying files imported from upstream: sed -ri ' s,(http|ftp)(://(.*\.)?(gnu|fsf|sourceware)\.org($|[^.]|\.[^a-z])),https\2,g s,(http|ftp)(://(.*\.)?)sources\.redhat\.com($|[^.]|\.[^a-z]),https\2sourceware.org\4,g ' \ $(find $(git ls-files) -prune -type f \ ! -name '*.po' \ ! -name 'ChangeLog*' \ ! -path COPYING ! -path COPYING.LIB \ ! -path manual/fdl-1.3.texi ! -path manual/lgpl-2.1.texi \ ! -path manual/texinfo.tex ! -path scripts/config.guess \ ! -path scripts/config.sub ! -path scripts/install-sh \ ! -path scripts/mkinstalldirs ! -path scripts/move-if-change \ ! -path INSTALL ! -path locale/programs/charmap-kw.h \ ! -path po/libc.pot ! -path sysdeps/gnu/errlist.c \ ! '(' -name configure \ -execdir test -f configure.ac -o -f configure.in ';' ')' \ ! '(' -name preconfigure \ -execdir test -f preconfigure.ac ';' ')' \ -print) and then by running 'make dist-prepare' to regenerate files built from the altered files, and then executing the following to cleanup: chmod a+x sysdeps/unix/sysv/linux/riscv/configure # Omit irrelevant whitespace and comment-only changes, # perhaps from a slightly-different Autoconf version. git checkout -f \ sysdeps/csky/configure \ sysdeps/hppa/configure \ sysdeps/riscv/configure \ sysdeps/unix/sysv/linux/csky/configure # Omit changes that caused a pre-commit check to fail like this: # remote: *** error: sysdeps/powerpc/powerpc64/ppc-mcount.S: trailing lines git checkout -f \ sysdeps/powerpc/powerpc64/ppc-mcount.S \ sysdeps/unix/sysv/linux/s390/s390-64/syscall.S # Omit change that caused a pre-commit check to fail like this: # remote: *** error: sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S: last line does not end in newline git checkout -f sysdeps/sparc/sparc64/multiarch/memcpy-ultra3.S
* Add glibc.malloc.mxfast tunableDJ Delorie2019-08-091-0/+2
| | | | | | | | | | | | * elf/dl-tunables.list: Add glibc.malloc.mxfast. * manual/tunables.texi: Document it. * malloc/malloc.c (do_set_mxfast): New. (__libc_mallopt): Call it. * malloc/arena.c: Add mxfast tunable. * malloc/tst-mxfast.c: New. * malloc/Makefile: Add it. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* Remove do_set_mallopt_check prototypeH.J. Lu2019-04-231-1/+0
| | | | | | | | | Remove do_set_mallopt_check prototype since it is unused. * malloc/arena.c (do_set_mallopt_check): Removed. Reviewed-by: Carlos O'Donell <carlos@redhat.com> Reviewed-by: DJ Delorie <dj@redhat.com>
* Update copyright dates with scripts/update-copyrights.Joseph Myers2019-01-011-1/+1
| | | | | | | * All files with FSF copyright notices: Update copyright dates using scripts/update-copyrights. * locale/programs/charmap-kw.h: Regenerated. * locale/programs/locfile-kw.h: Likewise.
* malloc: Convert the unlink macro to the unlink_chunk functionFlorian Weimer2018-11-121-2/+2
| | | | | | | This commit is in preparation of turning the macro into a proper function. The output arguments of the macro were in fact unused. Also clean up uses of __builtin_expect.
* Run thread shutdown functions in an explicit orderFlorian Weimer2018-06-261-3/+2
| | | | | | | This removes the __libc_thread_subfreeres hook in favor of explict calls. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* Update copyright dates with scripts/update-copyrights.Joseph Myers2018-01-011-1/+1
| | | | | | | * All files with FSF copyright notices: Update copyright dates using scripts/update-copyrights. * locale/programs/charmap-kw.h: Regenerated. * locale/programs/locfile-kw.h: Likewise.
* malloc: Call tcache destructor in arena_thread_freeresFlorian Weimer2017-11-231-0/+5
| | | | | | | | | It does not make sense to register separate cleanup functions for arena and tcache since they're always going to be called together. Call the tcache cleanup function from within arena_thread_freeres since it at least makes the order of those cleanups clear in the code. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
* Improve malloc initialization sequenceWilco Dijkstra2017-10-171-9/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | The current malloc initialization is quite convoluted. Instead of sometimes calling malloc_consolidate from ptmalloc_init, call malloc_init_state early so that the main_arena is always initialized. The special initialization can now be removed from malloc_consolidate. This also fixes BZ #22159. Check all calls to malloc_consolidate and remove calls that are redundant initialization after ptmalloc_init, like in int_mallinfo and __libc_mallopt (but keep the latter as consolidation is required for set_max_fast). Update comments to improve clarity. Remove impossible initialization check from _int_malloc, fix assert in do_check_malloc_state to ensure arena->top != 0. Fix the obvious bugs in do_check_free_chunk and do_check_remalloced_chunk to enable single threaded malloc debugging (do_check_malloc_state is not thread safe!). [BZ #22159] * malloc/arena.c (ptmalloc_init): Call malloc_init_state. * malloc/malloc.c (do_check_free_chunk): Fix build bug. (do_check_remalloced_chunk): Fix build bug. (do_check_malloc_state): Add assert that checks arena->top. (malloc_consolidate): Remove initialization. (int_mallinfo): Remove call to malloc_consolidate. (__libc_mallopt): Clarify why malloc_consolidate is needed.
* malloc: Remove the internal_function attributeFlorian Weimer2017-08-311-7/+0
|
* malloc: Remove corrupt arena flagFlorian Weimer2017-08-301-18/+2
| | | | | This is no longer needed because we now abort immediately once heap corruption is detected.
* malloc: Remove check_action variable [BZ #21754]Florian Weimer2017-08-301-8/+3
| | | | | | | | | | | Clean up calls to malloc_printerr and trim its argument list. This also removes a few bits of work done before calling malloc_printerr (such as unlocking operations). The tunable/environment variable still enables the lightweight additional malloc checking, but mallopt (M_CHECK_ACTION) no longer has any effect.
* Add per-thread cache to mallocDJ Delorie2017-07-061-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * config.make.in: Enable experimental malloc option. * configure.ac: Likewise. * configure: Regenerate. * manual/install.texi: Document it. * INSTALL: Regenerate. * malloc/Makefile: Likewise. * malloc/malloc.c: Add per-thread cache (tcache). (tcache_put): New. (tcache_get): New. (tcache_thread_freeres): New. (tcache_init): New. (__libc_malloc): Use cached chunks if available. (__libc_free): Initialize tcache if needed. (__libc_realloc): Likewise. (__libc_calloc): Likewise. (_int_malloc): Prefill tcache when appropriate. (_int_free): Likewise. (do_set_tcache_max): New. (do_set_tcache_count): New. (do_set_tcache_unsorted_limit): New. * manual/probes.texi: Document new probes. * malloc/arena.c: Add new tcache tunables. * elf/dl-tunables.list: Likewise. * manual/tunables.texi: Document them. * NEWS: Mention the per-thread cache.
* tunables: Clean up hooks to get and set tunablesSiddhesh Poyarekar2017-06-071-18/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The TUNABLE_SET_VALUE and family of macros (and my later attempt to add a TUNABLE_GET) never quite went together very well because the overall interface was not clearly defined. This patch is an attempt to do just that. This patch consolidates the API to two simple sets of macros, TUNABLE_GET* and TUNABLE_SET*. If TUNABLE_NAMESPACE is defined, TUNABLE_GET takes just the tunable name, type and a (optionally NULL) callback function to get the value of the tunable. The callback function, if non-NULL, is called if the tunable was externally set (i.e. via GLIBC_TUNABLES or any future mechanism). For example: val = TUNABLE_GET (check, int32_t, check_callback) returns the value of the glibc.malloc.check tunable (assuming TUNABLE_NAMESPACE is set to malloc) as an int32_t into VAL after calling check_callback. Likewise, TUNABLE_SET can be used to set the value of the tunable, although this is currently possible only in the dynamic linker before it relocates itself. For example: TUNABLE_SET (check, int32_t, 2) will set glibc.malloc.check to 2. Of course, this is not possible since we set (or read) glibc.malloc.check long after it is relocated. To access or set a tunable outside of TUNABLE_NAMESPACE, use the TUNABLE_GET_FULL and TUNABLE_SET_FULL macros, which have the following prototype: TUNABLE_GET_FULL (glibc, tune, hwcap_mask, uint64_t, NULL) TUNABLE_SET_FULL (glibc, tune, hwcap_mask, uint64_t, 0xffff) In future the tunable list may get split into mutable and immutable tunables where mutable tunables can be modified by the library and userspace after relocation as well and TUNABLE_SET will be more useful than it currently is. However whenever we actually do that split, we will have to ensure that the mutable tunables are protected with locks. * elf/Versions (__tunable_set_val): Rename to __tunable_get_val. * elf/dl-tunables.c: Likewise. (do_tunable_update_val): New function. (__tunable_set_val): New function. (__tunable_get_val): Call CB only if the tunable was externally initialized. (tunables_strtoul): Replace strval with initialized. * elf/dl-tunables.h (strval): Replace with a bool initialized. (TUNABLE_ENUM_NAME, TUNABLE_ENUM_NAME1): Adjust names to prevent collision. (__tunable_set_val): New function. (TUNABLE_GET, TUNABLE_GET_FULL): New macros. (TUNABLE_SET, TUNABLE_SET_FULL): Likewise. (TUNABLE_SET_VAL): Remove. (TUNABLE_SET_VAL_WITH_CALLBACK): Likewise. * README.tunables: Document the new macros. * malloc/arena.c (ptmalloc_init): Adjust.
* Fix getting tunable values on big-endian (BZ #21109)Siddhesh Poyarekar2017-02-081-4/+4
| | | | | | | The code to set value passed a tunable_val_t, which when cast to int32_t on big-endian gives the wrong value. Instead, use tunable_val_t.numval instead, which can then be safely cast into int32_t.
* Update copyright dates with scripts/update-copyrights.Joseph Myers2017-01-011-1/+1
|
* Add framework for tunablesSiddhesh Poyarekar2016-12-311-0/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The tunables framework allows us to uniformly manage and expose global variables inside glibc as switches to users. tunables/README has instructions for glibc developers to add new tunables. Tunables support can be enabled by passing the --enable-tunables configure flag to the configure script. This patch only adds a framework and does not pose any limitations on how tunable values are read from the user. It also adds environment variables used in malloc behaviour tweaking to the tunables framework as a PoC of the compatibility interface. * manual/install.texi: Add --enable-tunables option. * INSTALL: Regenerate. * README.tunables: New file. * Makeconfig (CPPFLAGS): Define TOP_NAMESPACE. (before-compile): Generate dl-tunable-list.h early. * config.h.in: Add HAVE_TUNABLES. * config.make.in: Add have-tunables. * configure.ac: Add --enable-tunables option. * configure: Regenerate. * csu/init-first.c (__libc_init_first): Move __libc_init_secure earlier... * csu/init-first.c (LIBC_START_MAIN):... to here. Include dl-tunables.h, libc-internal.h. (LIBC_START_MAIN) [!SHARED]: Initialize tunables for static binaries. * elf/Makefile (dl-routines): Add dl-tunables. * elf/Versions (ld): Add __tunable_set_val to GLIBC_PRIVATE namespace. * elf/dl-support (_dl_nondynamic_init): Unset MALLOC_CHECK_ only when !HAVE_TUNABLES. * elf/rtld.c (process_envvars): Likewise. * elf/dl-sysdep.c [HAVE_TUNABLES]: Include dl-tunables.h (_dl_sysdep_start): Call __tunables_init. * elf/dl-tunable-types.h: New file. * elf/dl-tunables.c: New file. * elf/dl-tunables.h: New file. * elf/dl-tunables.list: New file. * malloc/tst-malloc-usable-static.c: New test case. * malloc/Makefile (tests-static): Add it. * malloc/arena.c [HAVE_TUNABLES]: Include dl-tunables.h. Define TUNABLE_NAMESPACE. (DL_TUNABLE_CALLBACK (set_mallopt_check)): New function. (DL_TUNABLE_CALLBACK_FNDECL): New macro. Use it to define callback functions. (ptmalloc_init): Set tunable values. * scripts/gen-tunables.awk: New file. * sysdeps/mach/hurd/dl-sysdep.c: Include dl-tunables.h. (_dl_sysdep_start): Call __tunables_init.
* malloc: Use accessors for chunk metadata accessFlorian Weimer2016-10-281-3/+3
| | | | | This change allows us to change the encoding of these struct members in a centralized fashion.
* malloc: Manual part of conversion to __libc_lockFlorian Weimer2016-09-211-3/+3
| | | | | This removes the old mutex_t-related definitions from malloc-machine.h, too.
* malloc: Automated part of conversion to __libc_lockFlorian Weimer2016-09-061-26/+26
|
* malloc: Preserve arena free list/thread count invariant [BZ #20370]Florian Weimer2016-08-021-5/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | It is necessary to preserve the invariant that if an arena is on the free list, it has thread attach count zero. Otherwise, when arena_thread_freeres sees the zero attach count, it will add it, and without the invariant, an arena could get pushed to the list twice, resulting in a cycle. One possible execution trace looks like this: Thread 1 examines free list and observes it as empty. Thread 2 exits and adds its arena to the free list, with attached_threads == 0). Thread 1 selects this arena in reused_arena (not from the free list). Thread 1 increments attached_threads and attaches itself. (The arena remains on the free list.) Thread 1 exits, decrements attached_threads, and adds the arena to the free list. The final step creates a cycle in the usual way (by overwriting the next_free member with the former list head, while there is another list item pointing to the arena structure). tst-malloc-thread-exit exhibits this issue, but it was only visible with a debugger because the incorrect fix in bug 19243 removed the assert from get_free_list.
* malloc: Avoid premature fallback to mmap [BZ #20284]Florian Weimer2016-06-211-6/+4
| | | | | | | Before this change, the while loop in reused_arena which avoids returning a corrupt arena would never execute its body if the selected arena were not corrupt. As a result, result == begin after the loop, and the function returns NULL, triggering fallback to mmap.
* Revert __malloc_initialize_hook symbol poisoningFlorian Weimer2016-06-201-1/+1
| | | | | | It turns out the Emacs-internal malloc implementation uses __malloc_* symbols. If glibc poisons them in <stdc-pre.h>, Emacs will no longer compile.
* malloc: Remove __malloc_initialize_hook from the API [BZ #19564]Florian Weimer2016-06-101-1/+3
| | | | | | | __malloc_initialize_hook is interposed by application code, so the usual approach to define a compatibility symbol does not work. This commit adds a new mechanism based on #pragma GCC poison in <stdc-predef.h>.
* malloc: Add missing internal_function attributes on function definitionsFlorian Weimer2016-04-141-0/+3
| | | | Fixes build on i386 after commit 29d794863cd6e03115d3670707cc873a9965ba92.
* malloc: Remove malloc hooks from fork handlerFlorian Weimer2016-04-141-117/+8
| | | | | | | The fork handler now runs so late that there is no risk anymore that other fork handlers in the same thread use malloc, so it is no longer necessary to install malloc hooks which made a subset of malloc functionality available to the thread that called fork.
* malloc: Run fork handler as late as possible [BZ #19431]Florian Weimer2016-04-141-41/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, a thread M invoking fork would acquire locks in this order: (M1) malloc arena locks (in the registered fork handler) (M2) libio list lock A thread F invoking flush (NULL) would acquire locks in this order: (F1) libio list lock (F2) individual _IO_FILE locks A thread G running getdelim would use this order: (G1) _IO_FILE lock (G2) malloc arena lock After executing (M1), (F1), (G1), none of the threads can make progress. This commit changes the fork lock order to: (M'1) libio list lock (M'2) malloc arena locks It explicitly encodes the lock order in the implementations of fork, and does not rely on the registration order, thus avoiding the deadlock.
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-11-06 17:58:12 +0000