Shrink heap on linux when overcommit_memory == 2

Using madvise with MADV_DONTNEED to release memory back to the kernel
is not sufficient to change the commit charge accounted against the
process on Linux.  It is OK however, when overcommit is enabled or is
heuristic.  However, when overcommit is restricted to a percentage of
memory setting the contents of /proc/sys/vm/overcommit_memory as 2, it
makes a difference since memory requests will fail.  Hence, we do what
we do with secure exec binaries, which is to call mmap on the region
to be dropped with MAP_FIXED. This internally unmaps the pages in
question and reduces the amount of memory accounted against the
process.

1 files changed, 6 insertions, 3 deletions

diff --git a/malloc/arena.c b/malloc/arena.c
index f24e76c4f6..b209e3b7cd 100644
--- a/malloc/arena.c
+++ b/malloc/arena.c
@@ -19,6 +19,9 @@
 
 #include <stdbool.h>
 
+/* Get the implementation for check_may_shrink_heap.  */
+#include <malloc-sysdep.h>
+
 /* Compile-time constants.  */
 
 #define HEAP_MIN_SIZE (32*1024)
@@ -621,9 +624,9 @@ shrink_heap(heap_info *h, long diff)
   new_size = (long)h->size - diff;
   if(new_size < (long)sizeof(*h))
     return -1;
-  /* Try to re-map the extra heap space freshly to save memory, and
-     make it inaccessible. */
-  if (__builtin_expect (__libc_enable_secure, 0))
+  /* Try to re-map the extra heap space freshly to save memory, and make it
+     inaccessible.  See malloc-sysdep.h to know when this is true.  */
+  if (__builtin_expect (check_may_shrink_heap (), 0))
     {
       if((char *)MMAP((char *)h + new_size, diff, PROT_NONE,
 		      MAP_FIXED) == (char *) MAP_FAILED)