linux ttyname{_r}: Make tty checks consistent

In the ttyname and ttyname_r routines on Linux, at several points it needs to
check if a given TTY is the TTY we are looking for. It used to be that this
check was (to see if `maybe` is `mytty`):

       __xstat64(_STAT_VER, maybe_filename, &maybe) == 0
    #ifdef _STATBUF_ST_RDEV
       && S_ISCHR(maybe.st_mode) && maybe.st_rdev == mytty.st_rdev
    #else
       && maybe.st_ino == mytty.st_ino && maybe.st_dev == mytty.st_dev
    #endif

This check appears in several places.

Then, one of the changes made in commit 15e9a4f378c8607c2ae1aa465436af4321db0e23
was to change that check to:

       __xstat64(_STAT_VER, maybe_filename, &maybe) == 0
    #ifdef _STATBUF_ST_RDEV
       && S_ISCHR(maybe.st_mode) && maybe.st_rdev == mytty.st_rdev
    #endif
       && maybe.st_ino == mytty.st_ino && maybe.st_dev == mytty.st_dev

That is, it made the st_ino and st_dev parts of the check happen even if we have
the st_rdev member. This is an important change, because the kernel allows
multiple devpts filesystem instances to be created; a device file in one devpts
instance may share the same st_rdev with a file in another devpts instance, but
they aren't the same file.

This check appears twice in each file (ttyname.c and ttyname_r.c), once (in
ttyname and __ttyname_r) to check if a candidate file found by inspecting /proc
is the desired TTY, and once (in getttyname and getttyname_r) to check if a
candidate file found by searching /dev is the desired TTY. However, 15e9a4f
only updated the checks for files found via /proc; but the concern about
collisions between devpts instances is just as valid for files found via /dev.

So, update all 4 occurrences the check to be consistent with the version of the
check introduced in 15e9a4f. Make it easy to keep all 4 occurrences of the
check consistent by pulling it in to a static inline function, is_mytty.

Reviewed-by: Christian Brauner <christian.brauner@ubuntu.com>

1 files changed, 12 insertions, 0 deletions

diff --git a/sysdeps/unix/sysv/linux/ttyname.h b/sysdeps/unix/sysv/linux/ttyname.h
index cdaf60fb87..48181330a9 100644
--- a/sysdeps/unix/sysv/linux/ttyname.h
+++ b/sysdeps/unix/sysv/linux/ttyname.h
@@ -34,3 +34,15 @@ is_pty (struct stat64 *sb)
   return false;
 #endif
 }
+
+static inline bool
+is_mytty (const struct stat64 *mytty, const struct stat64 *maybe)
+{
+  return (maybe->st_ino == mytty->st_ino
+	  && maybe->st_dev == mytty->st_dev
+#ifdef _STATBUF_ST_RDEV
+	  && S_ISCHR (maybe->st_mode)
+	  && maybe->st_rdev == mytty->st_rdev
+#endif
+	  );
+}