gmon: improve mcount overflow handling [BZ# 27576]

When mcount overflows, no gmon.out file is generated, but no message is printed
to the user, leaving the user with no idea why, and thinking maybe there is
some bug - which is how BZ 27576 ended up being logged. Print a message to
stderr in this case so the user knows what is going on.

As a comment in sys/gmon.h acknowledges, the hardcoded MAXARCS value is too
small for some large applications, including the test case in that BZ. Rather
than increase it, add tunables to enable MINARCS and MAXARCS to be overridden
at runtime (glibc.gmon.minarcs and glibc.gmon.maxarcs). So if a user gets the
mcount overflow error, they can try increasing maxarcs (they might need to
increase minarcs too if the heuristic is wrong in their case.)

Note setting minarcs/maxarcs too large can cause monstartup to fail with an
out of memory error. If you set them large enough, it can cause an integer
overflow in calculating the buffer size. I haven't done anything to defend
against that - it would not generally be a security vulnerability, since these
tunables will be ignored in suid/sgid programs (due to the SXID_ERASE default),
and if you can set GLIBC_TUNABLES in the environment of a process, you can take
it over anyway (LD_PRELOAD, LD_LIBRARY_PATH, etc). I thought about modifying
the code of monstartup to defend against integer overflows, but doing so is
complicated, and I realise the existing code is susceptible to them even prior
to this change (e.g. try passing a pathologically large highpc argument to
monstartup), so I decided just to leave that possibility in-place.

Add a test case which demonstrates mcount overflow and the tunables.

Document the new tunables in the manual.

Signed-off-by: Simon Kissane <skissane@gmail.com>
Reviewed-by: DJ Delorie <dj@redhat.com>
(cherry picked from commit 31be941e4367c001b2009308839db5c67bf9dcbc)

1 files changed, 25 insertions, 4 deletions

diff --git a/gmon/gmon.c b/gmon/gmon.c
index bf76358d5b..689bf80141 100644
--- a/gmon/gmon.c
+++ b/gmon/gmon.c
@@ -46,6 +46,11 @@
 #include <libc-internal.h>
 #include <not-cancel.h>
 
+#if HAVE_TUNABLES
+# define TUNABLE_NAMESPACE gmon
+# include <elf/dl-tunables.h>
+#endif
+
 #ifdef PIC
 # include <link.h>
 
@@ -124,6 +129,22 @@ __monstartup (u_long lowpc, u_long highpc)
   int o;
   char *cp;
   struct gmonparam *p = &_gmonparam;
+  long int minarcs, maxarcs;
+
+#if HAVE_TUNABLES
+  /* Read minarcs/maxarcs tunables. */
+  minarcs = TUNABLE_GET (minarcs, int32_t, NULL);
+  maxarcs = TUNABLE_GET (maxarcs, int32_t, NULL);
+  if (maxarcs < minarcs)
+    {
+      ERR("monstartup: maxarcs < minarcs, setting maxarcs = minarcs\n");
+      maxarcs = minarcs;
+    }
+#else
+  /* No tunables, we use hardcoded defaults */
+  minarcs = MINARCS;
+  maxarcs = MAXARCS;
+#endif
 
   /*
    * round lowpc and highpc to multiples of the density we're using
@@ -146,10 +167,10 @@ __monstartup (u_long lowpc, u_long highpc)
   }
   p->fromssize = ROUNDUP(p->textsize / HASHFRACTION, sizeof(*p->froms));
   p->tolimit = p->textsize * ARCDENSITY / 100;
-  if (p->tolimit < MINARCS)
-    p->tolimit = MINARCS;
-  else if (p->tolimit > MAXARCS)
-    p->tolimit = MAXARCS;
+  if (p->tolimit < minarcs)
+    p->tolimit = minarcs;
+  else if (p->tolimit > maxarcs)
+    p->tolimit = maxarcs;
   p->tossize = p->tolimit * sizeof(struct tostruct);
 
   cp = calloc (p->kcountsize + p->fromssize + p->tossize, 1);