fix a64l undefined behavior on ILP32 archs, wrong results on LP64 archs

the difference of pointers is a signed type ptrdiff_t; if it is only
32-bit, left-shifting it by 30 bits produces undefined behavior. cast
the difference to an appropriate unsigned type, uint32_t, before
shifting to avoid this.

the a64l function is specified to return a signed 32-bit result in
type long. as noted in the bug report by Ed Schouten, converting
implicitly from uint32_t only produces the desired result when long is
a 32-bit type. since the computation has to be done in unsigned
arithmetic to avoid overflow, simply cast the result to int32_t.

further, POSIX leaves the behavior on invalid input unspecified but
not undefined, so we should not take the difference between the
potentially-null result of strchr and the base pointer without first
checking the result. the simplest behavior is just returning the
partial conversion already performed in this case, so do that.

1 files changed, 6 insertions, 3 deletions

diff --git a/src/misc/a64l.c b/src/misc/a64l.c
index 86aeefe0..60557710 100644
--- a/src/misc/a64l.c
+++ b/src/misc/a64l.c
@@ -9,9 +9,12 @@ long a64l(const char *s)
 {
 	int e;
 	uint32_t x = 0;
-	for (e=0; e<36 && *s; e+=6, s++)
-		x |= (strchr(digits, *s)-digits)<<e;
-	return x;
+	for (e=0; e<36 && *s; e+=6, s++) {
+		const char *d = strchr(digits, *s);
+		if (!d) break;
+		x |= (uint32_t)(d-digits)<<e;
+	}
+	return (int32_t)x;
 }
 
 char *l64a(long x0)