[fixed] Range coding error for floats

Index > Compiler Internals > [fixed] Range coding error for floats

Author

Thread

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 17717
Location: In your JS exploiting you and your system

revolution

fasm doesn't properly detect some invalid encodings.

When the exponent comes to either 0, or at maximum (all 1's), fasm generates some bad encodings.

eg.

Code:

;the qword set

 dq 3.5953862697246317e+308 ;out of range (correct)
 dq 3.5953862697246316e+308 ;QNaN (incorrect)
 dq 1.7976931348623162e+308 ;SNaN (incorrect)
 dq 1.7976931348623159e+308 ;infinity (incorrect)
 dq 1.7976931348623158e+308 ;good (correct)
 ......
 dq 2.2250738585072013e-308 ;good (correct)
 dq 1.6688053938804010e-308 ;denormal 1.1125369292536e-308 (incorrect)
 dq 1.1125369292536011e-308 ;denormal 9.8e-324 (incorrect)
 dq 1.1125369292536009e-308 ;denormal 4.9e-324 (incorrect)
 dq 1.1125369292536008e-308 ;0.0 (incorrect)
 dq 1.1125369292536006e-308 ;out of range (correct)


;the tbyte set

 dt 2.3794629907144635302e+4932 ;out of range (correct)
 dt 2.3794629907144635301e+4932 ;largest bad, high range (incorrect)
 dt 1.1897314953572317651e+4932 ;smallest bad, high range (incorrect)
 dt 1.1897314953572317650e+4932 ;good (correct)
 ......
 dt 3.3621031431120935062e-4932 ;good (correct)
 dt 3.3621031431120935061e-4932 ;largest bad, low range (incorrect)
 dt 1.6810515715560467531e-4932 ;smallest bad, low range (incorrect)
 dt 1.6810515715560467530e-4932 ;out of range (correct)

Last edited by revolution on 12 Feb 2009, 08:33; edited 1 time in total

02 Mar 2008, 16:18

vid
Verbosity in development

Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia

vid

good catch Smile

03 Mar 2008, 00:15

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 7802
Location: Kraków, Poland

Tomasz Grysztar

What are the exact value ranges that should be disallowed? I'm too lazy to check it myself. Wink

15 Nov 2008, 15:46

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 17717
Location: In your JS exploiting you and your system

revolution

Tomasz Grysztar wrote:

What are the exact value ranges that should be disallowed? I'm too lazy to check it myself.

For large numbers: If the exponent goes to all 1's then you have either an infinity or a NaN, so in those cases the assembler should ideally say number out of range.

For tiny numbers: If the exponent goes to all 0's then you have either a zero or a denormal. Denormals are not left justified and are stored with one extra shift to keep the MSb in the mantissa. When the mantissa is all 0's then the number is either + or - zero. If the mantissa does go to zero from a non zero decimal input then ideally the assembler should say number out of range.

19 Nov 2008, 14:25

bitRAKE

Joined: 21 Jul 2003
Posts: 3056
Location: vpcmipstrm

bitRAKE

Does this also effect the dword floats?

19 Nov 2008, 16:59

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 17717
Location: In your JS exploiting you and your system

revolution

bitRAKE wrote:

Does this also effect the dword floats?

I expect so. Just the ranges will change to reflect the smaller exponent range.

19 Nov 2008, 19:46

vid
Verbosity in development

Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia

vid

bitRAKE: FASM first converts to internal 3*32 bit mantissa, 32bit exponent (just like FASMLIB does, wonder why...), and then "downconverts" to desired format. So yeah, it definitively can cause problems with those too, but you have to be really "lucky" to hit it. But still I think this catch deserves Nobel prize for 2008 computing, really a great catch.

19 Nov 2008, 22:20

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 7802
Location: Kraków, Poland

Tomasz Grysztar

It's finally fixed in 1.67.31 (I think).

11 Feb 2009, 22:42

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 17717
Location: In your JS exploiting you and your system

revolution

Erm

qword conversion has a problem.

Code:

 dq 5e-324 ;2.147483647526668e-314  9C 2B 13 03 01 00 00 00

But the tbyte conversion works good. I haven't had a chance to test dword conversion.

12 Feb 2009, 03:22

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 7802
Location: Kraków, Poland

Tomasz Grysztar

Corrected.

12 Feb 2009, 07:35

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 17717
Location: In your JS exploiting you and your system

revolution

The rounding is not working for denormals. Maybe not important:

Code:

 dq 9.89e-324 ;9.88e-324 02 00 00 00 00 00 00 00
 dq 9.88e-324 ;4.94e-324 01 00 00 00 00 00 00 00

12 Feb 2009, 08:16

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 7802
Location: Kraków, Poland

Tomasz Grysztar

You're right, I didn't care to round them. Perhaps I will make it in a spare time.

12 Feb 2009, 08:27

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 7802
Location: Kraków, Poland

Tomasz Grysztar

Well, got some spare minutes right now. Smile

Can you check how much I broke it this time? Wink

12 Feb 2009, 09:41

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 17717
Location: In your JS exploiting you and your system

revolution

You get one star:

*

Congrats.

12 Feb 2009, 10:49

Madis731

Joined: 25 Sep 2003
Posts: 2140
Location: Estonia

Madis731

What does one star mean? Is it partly fixed or is a * a REALLY good thing ^o)

13 Feb 2009, 11:16

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 17717
Location: In your JS exploiting you and your system

revolution

Stars are VERY VERY hard to get. They don't come freely. If someone gets a star you can be sure it is for hard work and diligence above and beyond the normal requirement. So in this case, one star means "a REALLY good thing".

13 Feb 2009, 11:26

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum