about summary refs log tree commit diff
path: root/src/stdio/ftell.c
diff options
context:
space:
mode:
authorRich Felker <dalias@aerifal.cx>2014-03-09 03:09:49 -0400
committerRich Felker <dalias@aerifal.cx>2014-03-09 03:09:49 -0400
commit9743a399bf4d6da9a1dbdf7e8df07284c97df16f (patch)
treec9477850f2abba3b76123e1a2299b379199be9c4 /src/stdio/ftell.c
parentba231cf9e5923b6216081e9a626465c6643ce4d3 (diff)
downloadmusl-9743a399bf4d6da9a1dbdf7e8df07284c97df16f.tar.gz
musl-9743a399bf4d6da9a1dbdf7e8df07284c97df16f.tar.xz
musl-9743a399bf4d6da9a1dbdf7e8df07284c97df16f.zip
fix incorrect rounding in printf floating point corner cases
the printf floating point formatting code contains an optimization to
avoid computing digits that will be thrown away by rounding at the
specified (or default) precision. while it was correctly retaining all
places up to the last decimal place to be printed, it was not
retaining enough precision to see the next nonzero decimal place in
all cases. this could cause incorrect rounding down in round-to-even
(default) rounding mode, for example, when printing 0.5+DBL_EPSILON
with "%.0f".

in the fix, LDBL_MANT_DIG/3 is a lazy (non-sharp) upper bound on the
number of zeros between any two nonzero decimal digits.
Diffstat (limited to 'src/stdio/ftell.c')
0 files changed, 0 insertions, 0 deletions