about summary refs log tree commit diff
path: root/malloc
Commit message (Collapse)AuthorAgeFilesLines
* nptl: Move stack cache management, __libpthread_freeres into libcFlorian Weimer2021-05-111-5/+10
| | | | | | | | | | | | | This replaces the FREE_P macro with the __nptl_stack_in_use inline function. stack_list_del is renamed to __nptl_stack_list_del, stack_list_add to __nptl_stack_list_add, __deallocate_stack to __nptl_deallocate_stack, free_stacks to __nptl_free_stacks. It is convenient to move __libpthread_freeres into libc at the same time. This removes the temporary __default_pthread_attr_freeres export and restores full freeres coverage for __default_pthread_attr. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* malloc: Make tunable callback functions staticH.J. Lu2021-05-071-2/+2
| | | | | Since malloc tunable callback functions are only used within the same file, we should make them static.
* dlfcn: dlerror needs to call free from the base namespace [BZ #24773]Florian Weimer2021-04-212-7/+5
| | | | | | | | | | | | Calling free directly may end up freeing a pointer allocated by the dynamic loader using malloc from libc.so in the base namespace using the allocator from libc.so in a secondary namespace, which results in crashes. This commit redirects the free call through GLRO and the dynamic linker, to reach the correct namespace. It also cleans up the dlerror handling along the way, so that pthread_setspecific is no longer needed (which avoids triggering bug 24774).
* Further fixes for REALLOC_ZERO_BYTES_FREES commentPaul Eggert2021-04-121-7/+8
| | | | * malloc/malloc.c (REALLOC_ZERO_BYTES_FREES): Improve comment further.
* Fix REALLOC_ZERO_BYTES_FREES comment to match C17Paul Eggert2021-04-111-4/+7
| | | | | * malloc/malloc.c (REALLOC_ZERO_BYTES_FREES): Update comment to match current C standard.
* malloc: Run tst-malloc-stats-cancellation via test-driver.cArjun Shankar2021-04-071-2/+4
| | | | | | This allows the test to time out in case it hangs. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* malloc: Ensure mtag code path in checked_request2size is coldSzabolcs Nagy2021-03-261-2/+7
| | | | | | | | | This is a workaround (hack) for a gcc optimization issue (PR 99551). Without this the generated code may evaluate the expression in the cold path which causes performance regression for small allocations in the memory tagging disabled (common) case. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Remove unnecessary tagging around _mid_memalignSzabolcs Nagy2021-03-261-8/+2
| | | | | | | | The internal _mid_memalign already returns newly tagged memory. (__libc_memalign and posix_memalign already relied on this, this patch fixes the other call sites.) Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Rename chunk2rawmemSzabolcs Nagy2021-03-262-43/+43
| | | | | | | | | | | The previous patch ensured that all chunk to mem computations use chunk2rawmem, so now we can rename it to chunk2mem, and in the few cases where the tag of mem is relevant chunk2mem_tag can be used. Replaced tag_at (chunk2rawmem (x)) with chunk2mem_tag (x). Renamed chunk2rawmem to chunk2mem. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Use chunk2rawmem throughoutSzabolcs Nagy2021-03-262-26/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The difference between chunk2mem and chunk2rawmem is that the latter does not get the memory tag for the returned pointer. It turns out chunk2rawmem almost always works: The input of chunk2mem is a chunk pointer that is untagged so it can access the chunk header. All memory that is not user allocated heap memory is untagged, which in the current implementation means that it has the 0 tag, but this patch does not rely on the tag value. The patch relies on that chunk operations are either done on untagged chunks or without doing memory access to the user owned part. Internal interface contracts: sysmalloc: Returns untagged memory. _int_malloc: Returns untagged memory. _int_free: Takes untagged memory. _int_memalign: Returns untagged memory. _int_realloc: Takes and returns tagged memory. So only _int_realloc and functions outside this list need care. Alignment checks do not need the right tag and tcache works with untagged memory. tag_at was kept in realloc after an mremap, which is not strictly necessary, since the pointer is only used to retag the memory, but this way the tag is guaranteed to be different from the old tag. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Use different tag after mremapSzabolcs Nagy2021-03-261-1/+1
| | | | | | | | The comment explained why different tag is used after mremap, but for that correctly tagged pointer should be passed to tag_new_usable. Use chunk2mem to get the tag. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Use memsize instead of CHUNK_AVAILABLE_SIZESzabolcs Nagy2021-03-262-26/+24
| | | | | | | | | | | | | | | This is a pure refactoring change that does not affect behaviour. The CHUNK_AVAILABLE_SIZE name was unclear, the memsize name tries to follow the existing convention of mem denoting the allocation that is handed out to the user, while chunk is its internally used container. The user owned memory for a given chunk starts at chunk2mem(p) and the size is memsize(p). It is not valid to use on dumped heap chunks. Moved the definition next to other chunk and mem related macros. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Use mtag_enabled instead of USE_MTAGSzabolcs Nagy2021-03-262-12/+8
| | | | | | | | | Use the runtime check where possible: it should not cause slow down in the !USE_MTAG case since then mtag_enabled is constant false, but it allows compiling the tagging logic so it's less likely to break or diverge when developers only test the !USE_MTAG case. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Use branches instead of mtag_granule_maskSzabolcs Nagy2021-03-262-21/+14
| | | | | | | | | | | | | The branches may be better optimized since mtag_enabled is widely used. Granule size larger than a chunk header is not supported since then we cannot have both the chunk header and user area granule aligned. To fix that for targets with large granule, the chunk layout has to change. So code that attempted to handle the granule mask generally was changed. This simplified CHUNK_AVAILABLE_SIZE and the logic in malloc_usable_size. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Change calloc when tagging is disabledSzabolcs Nagy2021-03-261-6/+4
| | | | | | | | | | | | When glibc is built with memory tagging support (USE_MTAG) but it is not enabled at runtime (mtag_enabled) then unconditional memset was used even though that can be often avoided. This is for performance when tagging is supported but not enabled. The extra check should have no overhead: tag_new_zero_region already had a runtime check which the compiler can now optimize away. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Only support zeroing and not arbitrary memset with mtagSzabolcs Nagy2021-03-261-9/+8
| | | | | | | | | | The memset api is suboptimal and does not provide much benefit. Memory tagging only needs a zeroing memset (and only for memory that's sized and aligned to multiples of the tag granule), so change the internal api and the target hooks accordingly. This is to simplify the implementation of the target hook. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Use global flag instead of function pointer dispatch for mtagSzabolcs Nagy2021-03-262-52/+39
| | | | | | | | | | | | | | | | | | A flag check can be faster than function pointers because of how branch prediction and speculation works and it can also remove a layer of indirection when there is a mismatch between the malloc internal tag_* api and __libc_mtag_* target hooks. Memory tagging wrapper functions are moved to malloc.c from arena.c and the logic now checks mmap_enabled. The definition of tag_new_usable is moved after chunk related definitions. This refactoring also allows using mtag_enabled checks instead of USE_MTAG ifdefs when memory tagging support only changes code logic when memory tagging is enabled at runtime. Note: an "if (false)" code block is optimized away even at -O0 by gcc. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Refactor TAG_ macros to avoid indirectionSzabolcs Nagy2021-03-263-56/+51
| | | | | | | | | | | This does not change behaviour, just removes one layer of indirection in the internal memory tagging logic. Use tag_ and mtag_ prefixes instead of __tag_ and __mtag_ since these are all symbols with internal linkage, private to malloc.c, so there is no user namespace pollution issue. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Avoid taggig mmaped memory on freeSzabolcs Nagy2021-03-261-3/+4
| | | | | | | | | | Either the memory belongs to the dumped area, in which case we don't want to tag (the dumped area has the same tag as malloc internal data so tagging is unnecessary, but chunks there may not have the right alignment for the tag granule), or the memory will be unmapped immediately (and thus tagging is not useful). Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Simplify __mtag_tag_new_usableSzabolcs Nagy2021-03-261-5/+0
| | | | | | | | | The chunk cannot be a dumped one here. The only non-obvious cases are free and realloc which may be called on a dumped area chunk, but in both cases it can be verified that tagging is already avoided for dumped area chunks. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Move MTAG_MMAP_FLAGS definitionSzabolcs Nagy2021-03-261-0/+2
| | | | | | | This is only used internally in malloc.c, the extern declaration was wrong, __mtag_mmap_flags has internal linkage. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Fix a potential realloc issue with memory taggingSzabolcs Nagy2021-03-261-7/+7
| | | | | | | | | | | | At an _int_free call site in realloc the wrong size was used for tag clearing: the chunk header of the next chunk was also cleared which in practice may work, but logically wrong. The tag clearing is moved before the memcpy to save a tag computation, this avoids a chunk2mem. Another chunk2mem is removed because newmem does not have to be recomputed. Whitespaces got fixed too. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Fix a realloc crash with heap tagging [BZ 27468]Szabolcs Nagy2021-03-261-1/+3
| | | | | | | | | | | | _int_free must be called with a chunk that has its tag reset. This was missing in a rare case that could crash when heap tagging is enabled: when in a multi-threaded process the current arena runs out of memory during realloc, but another arena still has space to finish the realloc then _int_free was called without clearing the user allocation tags. Fixes bug 27468. Reviewed-by: DJ Delorie <dj@redhat.com>
* Support for multiple versions in versioned_symbol, compat_symbolFlorian Weimer2021-03-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This essentially folds compat_symbol_unique functionality into compat_symbol. This change eliminates the need for intermediate aliases for defining multiple symbol versions, for both compat_symbol and versioned_symbol. Some binutils versions do not suport multiple versions per symbol on some targets, so aliases are automatically introduced, similar to what compat_symbol_unique did. To reduce symbol table sizes, a configure check is added to avoid these aliases if they are not needed. The new mechanism works with data symbols as well as function symbols, due to the way an assembler-level redirect is used. It is not compatible with weak symbols for old binutils versions, which is why the definition of __malloc_initialize_hook had to be changed. This is not a loss of functionality because weak symbols do not matter to dynamic linking. The placeholder symbol needs repeating in nptl/libpthread-compat.c now that compat_symbol is used, but that seems more obvious than introducing yet another macro. A subtle difference was that compat_symbol_unique made the symbol global automatically. compat_symbol does not do this, so static had to be removed from the definition of __libpthread_version_placeholder. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* malloc: Turn tst-mallocstate into a non-internal testFlorian Weimer2021-03-092-13/+8
| | | | | | | | | compat_symbol_reference no longer needs tests-internal. Do not build the test at all for newer targets, so that no spurious UNSUPPORTED result is generated. Use compat_symbol_reference for __malloc_initialize_hook as well, eliminating the need for -rdynamic. Reviewed-by: DJ Delorie <dj@redhat.com>
* Implement <unwind-link.h> for dynamically loading the libgcc_s unwinderFlorian Weimer2021-03-011-0/+5
| | | | | | | | | | This will be used to consolidate the libgcc_s access for backtrace and pthread_cancel. Unlike the existing backtrace implementations, it provides some hardening based on pointer mangling. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* added rt to malloc/Depend [BZ #27132]Paul Zimmermann2021-02-231-0/+1
| | | | | This avoids a failure when a new glibc version is compiled with an older system librt. Patch proposed by Florian Weimer.
* malloc: Sync dynarray with gnulibAdhemerval Zanella2021-02-099-84/+95
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It syncs with gnulib version a8bac4d49. The main changes are: - Remove the usage of anonymous union within DYNARRAY_STRUCT. - Use DYNARRAY_FREE instead of DYNARRAY_NAME (free) so that Gnulib does not change 'free' to 'rpl_free'. - Use __nonnull instead of __attribute__ ((nonnull ())). - Use __attribute_maybe_unused__ instead of __attribute__ ((unused, nonnull (1))). - Use of _Noreturn instead of _attribute__ ((noreturn)). The only difference with gnulib is: --- glibc +++ gnulib @@ -18,6 +18,7 @@ #include <dynarray.h> #include <stdio.h> +#include <stdlib.h> void __libc_dynarray_at_failure (size_t size, size_t index) @@ -27,7 +28,6 @@ __snprintf (buf, sizeof (buf), "Fatal glibc error: " "array index %zu not less than array length %zu\n", index, size); - __libc_fatal (buf); #else abort (); #endif It seems a wrong sync from gnulib (the code is used on loader and thus it requires __libc_fatal instead of abort). Checked on x86_64-linux-gnu.
* tst-mallinfo2.c: Remove useless trailing semicolon for macroYang Xu2021-02-011-1/+1
| | | | | | Macros should not use a trailing semicolon, so remove it. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Add scratch_buffer_dupfreeAdhemerval Zanella2021-01-054-2/+67
| | | | | It returns a copy of the buffer up to a defined size. It will be used on realpath sync with gnulib.
* Update copyright dates not handled by scripts/update-copyrights.Paul Eggert2021-01-023-3/+3
| | | | | | | | | | | | | | I've updated copyright dates in glibc for 2021. This is the patch for the changes not generated by scripts/update-copyrights and subsequent build / regeneration of generated files. As well as the usual annual updates, mainly dates in --version output (minus csu/version.c which previously had to be handled manually but is now successfully updated by update-copyrights), there is a small change to the copyright notice in NEWS which should let NEWS get updated automatically next year. Please remember to include 2021 in the dates for any new files added in future (which means updating any existing uncommitted patches you have that add new files to use the new copyright dates in them).
* Update copyright dates with scripts/update-copyrightsPaul Eggert2021-01-0281-81/+81
| | | | | | | | | | | | | | | | I used these shell commands: ../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright (cd ../glibc && git commit -am"[this commit message]") and then ignored the output, which consisted lines saying "FOO: warning: copyright statement not found" for each of 6694 files FOO. I then removed trailing white space from benchtests/bench-pthread-locks.c and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this diagnostic from Savannah: remote: *** pre-commit check failed ... remote: *** error: lines with trailing whitespace found remote: error: hook declined to update refs/heads/master
* malloc: preserve errno on mcheck hooks [BZ #17924]Adhemerval Zanella2020-12-301-5/+11
| | | | | | | | | Similar to the fix 69fda43b8d, save and restore errno for the hook functions used for MALLOC_CHECK_=3. It fixes the malloc/tst-free-errno-mcheck regression. Checked on x86_64-linux-gnu.
* free: preserve errno [BZ#17924]Paul Eggert2020-12-293-4/+141
| | | | | | | | | | | In the next release of POSIX, free must preserve errno <https://www.austingroupbugs.net/view.php?id=385>. Modify __libc_free to save and restore errno, so that any internal munmap etc. syscalls do not disturb the caller's errno. Add a test malloc/tst-free-errno.c (almost all by Bruno Haible), and document that free preserves errno. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* MTE: Do not pad size in realloc_checkSiddhesh Poyarekar2020-12-241-4/+5
| | | | | | | | | | The MTE patch to add malloc support incorrectly padded the size passed to _int_realloc by SIZE_SZ when it ought to have sent just the chunksize. Revert that bit of the change so that realloc works correctly with MALLOC_CHECK_ set. This also brings the realloc_check implementation back in sync with libc_realloc.
* tests-mcheck: New variable to run tests with MALLOC_CHECK_=3Siddhesh Poyarekar2020-12-241-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | This new variable allows various subsystems in glibc to run all or some of their tests with MALLOC_CHECK_=3. This patch adds infrastructure support for this variable as well as an implementation in malloc/Makefile to allow running some of the tests with MALLOC_CHECK_=3. At present some tests in malloc/ have been excluded from the mcheck tests either because they're specifically testing MALLOC_CHECK_ or they are failing in master even without the Memory Tagging patches that prompted this work. Some tests were reviewed and found to need specific error points that MALLOC_CHECK_ defeats by terminating early but a thorough review of all tests is needed to bring them into mcheck coverage. The following failures are seen in current master: FAIL: malloc/tst-malloc-fork-deadlock-mcheck FAIL: malloc/tst-malloc-stats-cancellation-mcheck FAIL: malloc/tst-malloc-thread-fail-mcheck FAIL: malloc/tst-realloc-mcheck FAIL: malloc/tst-reallocarray-mcheck All of these are due to the Memory Tagging patchset and will be fixed separately.
* malloc: Basic support for memory tagging in the malloc() familyRichard Earnshaw2020-12-213-97/+377
| | | | | | | | | | | | | | | | | This patch adds the basic support for memory tagging. Various flavours are supported, particularly being able to turn on tagged memory at run-time: this allows the same code to be used on systems where memory tagging support is not present without neededing a separate build of glibc. Also, depending on whether the kernel supports it, the code will use mmap for the default arena if morecore does not, or cannot support tagged memory (on AArch64 it is not available). All the hooks use function pointers to allow this to work without needing ifuncs. Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc: Use __libc_initial to detect an inner libcFlorian Weimer2020-12-162-8/+7
| | | | | | | | | | | | | The secondary/non-primary/inner libc (loaded via dlmopen, LD_AUDIT, static dlopen) must not use sbrk to allocate member because that would interfere with allocations in the outer libc. On Linux, this does not matter because sbrk itself was changed to fail in secondary libcs. _dl_addr occasionally shows up in profiles, but had to be used before because __libc_multiple_libs was unreliable. So this change achieves a slight reduction in startup time. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* malloc: Detect infinite-loop in _int_free when freeing tcache [BZ#27052]W. Hashimoto2020-12-111-1/+4
| | | | | | | | | | | If linked-list of tcache contains a loop, it invokes infinite loop in _int_free when freeing tcache. The PoC which invokes such infinite loop is on the Bugzilla(#27052). This loop should terminate when the loop exceeds mp_.tcache_count and the program should abort. The affected glibc version is 2.29 or later. Reviewed-by: DJ Delorie <dj@redhat.com>
* nsswitch: use new internal API (core)DJ Delorie2020-12-041-1/+3
| | | | | | Core changes to switch the NSS internals to use the new API. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
* nss: Introduce <nss_module.h>Florian Weimer2020-12-041-0/+3
| | | | | | | | | | | | | | | | This provides the struct nss_module type, which combines the old struct service_library type with the known_function tree, by statically allocating space for all function pointers. struct nss_module is fairly large (536 bytes), but it will be shared across NSS databases. The old known_function handling had non-some per-function overhead (at least 32 bytes per looked-up function, but more for long function anmes), so overall, this is not too bad. Resolving all functions at load time simplifies locking, and the repeated lookups should be fast because the caches are hot at this point. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
* Remove tls.h inclusion from internal errno.hAdhemerval Zanella2020-11-131-0/+1
| | | | | | | | | | | | The tls.h inclusion is not really required and limits possible definition on more arch specific headers. This is a cleanup to allow inline functions on sysdep.h, more specifically on i386 and ia64 which requires to access some tls definitions its own. No semantic changes expected, checked with a build against all affected ABIs.
* malloc debug: fix compile error when enable macro MALLOC_DEBUG > 1liqingqing2020-10-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | malloc debug: fix compile error when enable macro MALLOC_DEBUG > 1. this is because commit e9c4fe93b3855239752819303ca377dff0ed0553 has change the struct malloc_chunk's member "size" to "mchunk_size". the reproduction is like that: setp1: modify related Makefile. vim ../glibc/malloc/Makefile CPPFLAGS-malloc.o += -DMALLOC_DEBUG=2 step2: ../configure --prefix=/usr make -j32 this will cause the compile error: /home/liqingqing/glibc_upstream/buildglibc/malloc/malloc.o In file included from malloc.c:1899:0: arena.c: In function 'dump_heap': arena.c:422:58: error: 'struct malloc_chunk' has no member named 'size' fprintf (stderr, "chunk %p size %10lx", p, (long) p->size); ^~ arena.c:428:17: error: 'struct malloc_chunk' has no member named 'size' else if (p->size == (0 | PREV_INUSE)) Reviewed-by: DJ Delorie <dj@redhat.com>
* tst-tcfree2: adjust coding style.liqingqing2020-10-301-4/+4
| | | | | | tst-tcfree2: adjust coding style. Reviewed-by: DJ Delorie <dj@redhat.com>
* Revert "Fix missing redirects in testsuite targets"Andreas Schwab2020-10-081-1/+1
| | | | | This reverts commit d5afb38503. The log files are actually created by the various shell scripts that drive the tests.
* Replace Minumum/minumum with Minimum/minimumH.J. Lu2020-10-061-1/+1
| | | | Replace Minumum/minumum in comments with Minimum/minimum.
* Update mallinfo2 ABI, and testDJ Delorie2020-09-174-1/+89
| | | | | | | This patch adds the ABI-related bits to reflect the new mallinfo2 function, and adds a test case to verify basic functionality. Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
* malloc: Fix mallinfo deprecation declarationAdhemerval Zanella2020-08-313-2/+15
| | | | | | | | | | | | | | | | It fixes the build issue below introduced by e3960d1c57e57 (Add mallinfo2 function that support sizes >= 4GB). It moves the __MALLOC_DEPRECATED to the usual place for function attributes: In file included from ../include/malloc.h:3, from ../sysdeps/x86_64/multiarch/../../../test-skeleton.c:31, from ../sysdeps/x86_64/multiarch/test-multiarch.c:96: ../malloc/malloc.h:118:1: error: empty declaration [-Werror] 118 | __MALLOC_DEPRECATED; It also adds the required deprecated warning suppression on the tests. Checked on x86_64-linux-gnu.
* Add mallinfo2 function that support sizes >= 4GB.Martin Liska2020-08-312-5/+51
| | | | | The current int type can easily overflow for allocation of more than 4GB.
* Remove --enable-obsolete-rpc configure flagPetr Vorel2020-07-131-1/+4
| | | | | | | | | | | | | | | | | | | | | | Sun RPC was removed from glibc. This includes rpcgen program, librpcsvc, and Sun RPC headers. Also test for bug #20790 was removed (test for rpcgen). Backward compatibility for old programs is kept only for architectures and ABIs that have been added in or before version 2.28. libtirpc is mature enough, librpcsvc and rpcgen are provided in rpcsvc-proto project. NOTE: libnsl code depends on Sun RPC (installed libnsl headers use installed Sun RPC headers), thus --enable-obsolete-rpc was a dependency for --enable-obsolete-nsl (removed in a previous commit). The arc ABI list file has to be updated because the port was added with the sunrpc symbols Tested-by: Carlos O'Donell <carlos@redhat.com> Reviewed-by: Carlos O'Donell <carlos@redhat.com>