MIPS: Add wrappers to get/setrlimit64 to fix RLIM64_INFINITY constant

RLIM64_INFINITY was supposed to be a glibc convention rather than
anything seen by the kernel, but it ended being passed to the kernel
through the prlimit64 syscall.  On O32 and N32 ABIs, we therefore
end-up with different values on the userland and kernel side:

* On the kernel side, the value is defined for all architectures as
  include/uapi/linux/resource.h:

  #define RLIM64_INFINITY           (~0ULL)

* On the GNU libc side, the value is defined in
  ports/sysdeps/unix/sysv/linux/mips/bits/resource.h:

  For the O32 and N32 ABI:
  #  define RLIM64_INFINITY 0x7fffffffffffffffULL

  and for the N64 ABI:
  #  define RLIM64_INFINITY 0xffffffffffffffffUL

This was not a problem until the prlimit64 syscall was wired in the
2.6.36 kernel. Given the GLIBC uses the prlimit64 syscall to implement
getrlimit64 and setrlimit64, pam_limits.so is setting the limits to
a very big value instead of infinity. As a normal user process can
later only decrease the value and not increase it, it will later get
and EPERM error when trying to set the value to infinity with setrlimit.

The GLIBC has this constant for more than 7 years, and as it is defined
in a header file, it means a lot of binaries are in the wild. This patch
fixes that by adding a wrapper to fix the value passed to or received
from the kernel, before or after calling the prlimit64 syscall.

1 files changed, 8 insertions, 0 deletions

diff --git a/ports/ChangeLog.mips b/ports/ChangeLog.mips
index 753e04b908..d2212a1933 100644
--- a/ports/ChangeLog.mips
+++ b/ports/ChangeLog.mips
@@ -1,3 +1,11 @@
+2013-11-27  Aurelien Jarno <aurelien@aurel32.net>
+
+	* sysdeps/unix/sysv/linux/mips/getrlimit64.c: On O32 and N32 ABIs,
+	include the generic getrlimit64 version as __internal_getrlimit64
+	and add a wrapper around it to convert the RLIM64_INFINITY constant
+	between the libc and the kernel version.
+	* sysdeps/unix/sysv/linux/mips/setrlimit64.c: Ditto with setrlimit64.
+
 2013-11-26  Ondřej Bílka  <neleai@seznam.cz>
 	* sysdeps/unix/sysv/linux/mips/bits/ipc.h: Use __glibc_reserved instead __unused.
 	* sysdeps/unix/sysv/linux/mips/bits/msq.h: Likewise.