fix popen not to leak pipes from one child to another

POSIX places an obscure requirement on popen which is like a limited
version of close-on-exec:

    "The popen() function shall ensure that any streams from previous
    popen() calls that remain open in the parent process are closed in
    the new child process."

if the POSIX-future 'e' mode flag is passed, producing a pipe FILE
with FD_CLOEXEC on the underlying pipe, this requirement is
automatically satisfied. however, for applications which use multiple
concurrent popen pipes but don't request close-on-exec, fd leaks from
earlier popen calls to later ones could produce deadlock situations
where processes are waiting for a pipe EOF that will never happen.

to fix this, iterate through all open FILEs and add close actions for
those obtained from popen. this requires holding a lock on the open
file list across the posix_spawn call so that additional popen FILEs
are not created after the list is traversed. note that it's still
possible for another popen call to start and create its pipe while the
lock is held, but such pipes are created with O_CLOEXEC and only drop
close-on-exec status (when 'e' flag is omitted) under control of the
lock.

1 files changed, 6 insertions, 0 deletions

diff --git a/src/stdio/popen.c b/src/stdio/popen.c
index 19bbe7cf..3ec83394 100644
--- a/src/stdio/popen.c
+++ b/src/stdio/popen.c
@@ -34,6 +34,9 @@ FILE *popen(const char *cmd, const char *mode)
 
 	e = ENOMEM;
 	if (!posix_spawn_file_actions_init(&fa)) {
+		for (FILE *l = *__ofl_lock(); l; l=l->next)
+			if (l->pipe_pid && posix_spawn_file_actions_addclose(&fa, l->fd))
+				goto fail;
 		if (!posix_spawn_file_actions_adddup2(&fa, p[1-op], 1-op)) {
 			if (!(e = posix_spawn(&pid, "/bin/sh", &fa, 0,
 			    (char *[]){ "sh", "-c", (char *)cmd, 0 }, __environ))) {
@@ -42,9 +45,12 @@ FILE *popen(const char *cmd, const char *mode)
 				if (!strchr(mode, 'e'))
 					fcntl(p[op], F_SETFD, 0);
 				__syscall(SYS_close, p[1-op]);
+				__ofl_unlock();
 				return f;
 			}
 		}
+fail:
+		__ofl_unlock();
 		posix_spawn_file_actions_destroy(&fa);
 	}
 	fclose(f);