diff options
author | Rich Felker <dalias@aerifal.cx> | 2018-04-11 15:05:22 -0400 |
---|---|---|
committer | Rich Felker <dalias@aerifal.cx> | 2018-04-11 15:05:22 -0400 |
commit | 10e4bd3780050e75b72aac5d85c31816419bb17d (patch) | |
tree | 45bda2ae5bd8abb668dd8db53365b488c293f8c3 /src/math/fmaxf.c | |
parent | ae2a01da2e388535da243b3d974aef74a3c06ae0 (diff) | |
download | musl-10e4bd3780050e75b72aac5d85c31816419bb17d.tar.gz musl-10e4bd3780050e75b72aac5d85c31816419bb17d.tar.xz musl-10e4bd3780050e75b72aac5d85c31816419bb17d.zip |
fix incorrect results for catan with some inputs
the catan implementation from OpenBSD includes a FIXME-annotated "overflow" branch that produces a meaningless and incorrect large-magnitude result. it was reachable via three paths, corresponding to gotos removed by this commit, in order: 1. pure imaginary argument with imaginary component greater than 1 in magnitude. this case does not seem at all exceptional and is handled (at least with the quality currently expected from our complex math functions) by the existing non-exceptional code path. 2. arguments on the unit circle, including the pure-real argument 1.0. these are not exceptional except for ±i, which should produce results with infinite imaginary component and which lead to computation of atan2(±0,0) in the existing non-exceptional code path. such calls to atan2() however are well-defined by POSIX. 3. the specific argument +i. this route should be unreachable due to the above (2), but subtle rounding effects might have made it possible in rare cases. continuing on the non-exceptional code path in this case would lead to computing the (real) log of an infinite argument, then producing a NAN when multiplying it by I. for now, remove the exceptional code paths entirely. replace the multiplication by I with construction of a complex number using the CMPLX macro so that the NAN issue (3) prevented cannot arise. with these changes, catan should give reasonably correct results for real arguments, and should no longer give completely-wrong results for pure-imaginary arguments outside the interval (-i,+i).
Diffstat (limited to 'src/math/fmaxf.c')
0 files changed, 0 insertions, 0 deletions