I have written a string to ,half float, float, double and extended double, conversion routine as a class project. I was testing it against the floating point converters in flat assembler when I found a small difference in the way flat assembler handles the under flow condition and the way my Intel processor handled it.
If the exponent is zero and the significand is zero flat assembler will issue an out of range error. I think this test is performed before the floating point number is rounded. There are a few cases where rounding will cause the significand to round up to 1, which should prevent the out of range message.
;---------
dw 5.959E-8 ;causes an out of range error
I created a float:
dd 5.959e-8 ;This float was converted to half precision using VCVTPS2PH with
the round code set to 0. One will get a half precision value of 0x0001
;---------
dd 1.4012984220e-45 ;causes an out of range error
I created a double:
dq 1.4012984220e-45 ;this double was converted to a float using CVTPD2PS with the flags set for rounding. The floating point value created by the conversion was 0x00000001
;----------
dq 4.940656458e-324 ;will cause an out of range error. With rounding set it should convert to 0x0000000000000001
;------------------
|