Fix ldbl-128 / ldbl-128ibm lgamma overflow handling (bug 16347, bug 19046).

The ldbl-128 / ldbl-128ibm implementation of lgamma has problems with
its handling of large arguments.  It has an overflow threshold that is
correct only for ldbl-128, despite being used for both types - with
diagnostic control macros as a temporary measure to disable warnings
about that constant overflowing for ldbl-128ibm - and it has a
calculation that's roughly x * log(x) - x, resulting in overflows for
arguments that are roughly at most a factor 1/log(threshold) below the
overflow threshold.

This patch fixes both issues, using an overflow threshold appropriate
for the type in question and adding another case for large arguments
that avoids the possible intermediate overflow.

Tested for x86_64, x86, mips64 and powerpc.

	[BZ #16347]
	[BZ #19046]
	* sysdeps/ieee754/ldbl-128/e_lgammal_r.c: Do not include
	<libc-internal.h>.
	(MAXLGM): Do not use diagnostic control macros.
	[LDBL_MANT_DIG == 106] (MAXLGM): Change value to overflow
	threshold for ldbl-128ibm.
	(__ieee754_lgammal_r): For large arguments, multiply by log - 1
	instead of multiplying by log then subtracting.
	* math/auto-libm-test-in: Add more tests of lgamma.
	* math/auto-libm-test-out: Regenerated.

1 files changed, 18 insertions, 0 deletions

diff --git a/math/auto-libm-test-in b/math/auto-libm-test-in
index 25d6b9df16..3399be1a64 100644
--- a/math/auto-libm-test-in
+++ b/math/auto-libm-test-in
@@ -2505,6 +2505,24 @@ lgamma -0x1p-16445
 lgamma 0x1p-16494
 lgamma -0x1p-16494
 
+# Values +/- 10ulp from overflow threshold.  (Values very close to
+# overflow threshold produce results very close of that threshold,
+# where a result inaccurate by a few ulp could differ from the ideal
+# result in whether it overflows; +/- 10ulp is sufficient for overflow
+# or its absence to be unambiguous under glibc's accuracy standards).
+# This also means the ldbl-128ibm inputs are XFAILed for dbl-64 and
+# the ldbl-128 inputs for ldbl-96, as too close to the threshold.
+lgamma 0x3.12be0cp+120
+lgamma 0x3.12be6p+120
+lgamma 0x5.d53649e2d4674p+1012
+lgamma 0x5.d53649e2d46c8p+1012
+lgamma 0x5.d53649e2d469dbc1f01e99fd52p+1012 xfail:dbl-64
+lgamma 0x5.d53649e2d469dbc1f01e99fd7cp+1012 xfail:dbl-64
+lgamma 0x5.c6aa645fffef5f5p+16368
+lgamma 0x5.c6aa645fffef5ff8p+16368
+lgamma 0x5.c6aa645fffef5fa912b9b480f7acp+16368 xfail:ldbl-96-intel xfail:ldbl-96-m68k
+lgamma 0x5.c6aa645fffef5fa912b9b480f8p+16368 xfail:ldbl-96-intel xfail:ldbl-96-m68k
+
 lgamma -0x1.fa471547c2fe5p+1
 lgamma -0x1.9260dcp+1