flat assembler
Message board for the users of flat assembler.

 Index > Main > Why cant I add floating point number right
Author
NanoBytes

Joined: 02 Jun 2011
Posts: 57
Location: Iowa, United States
NanoBytes 02 Feb 2012, 03:09
I am trying to add 12 to 0.2, but it keeps returning 12.199... and I cant figure out how to get it to work

Code:
```        mov [Integer1],12
FILD [Integer1]
mov [Integer1],2
FILD [Integer1]
MOV [Integer1],10
FIDIV [Integer1]
```

_________________
He is no fool who gives what he cannot
keep to gain what he cannot loose.
02 Feb 2012, 03:09
revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 19076
revolution 02 Feb 2012, 03:20
12.2 cannot be perfectly represented as a floating point number. The FPU will use the closest approximation available 12.199999... Generally one would round the numbers to the required precision for display and the small error in the internal representation would not be seen.
02 Feb 2012, 03:20
NanoBytes

Joined: 02 Jun 2011
Posts: 57
Location: Iowa, United States
NanoBytes 02 Feb 2012, 03:39
Ahh, i think i figured it out
Code:
```        MOV [Integer1],15
FILD [Integer1]
MOV [Integer1],2
FILD [Integer1]
MOV [Integer1],10
FIDIV [Integer1]
FSUBP st1,st0
FSTP [Integer1]
BTS [Integer1],2
FLD [Integer1]
```

Though, i am worried, I am converting from the registers 10 bytes to the integers 4 bytes just to correctly add two numbers. Is there a more accurate way to do this?

_________________
He is no fool who gives what he cannot
keep to gain what he cannot loose.
02 Feb 2012, 03:39
NanoBytes

Joined: 02 Jun 2011
Posts: 57
Location: Iowa, United States
NanoBytes 02 Feb 2012, 04:01
Ok, revolution, how would i round the entire number off?
02 Feb 2012, 04:01
revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 19076
revolution 02 Feb 2012, 04:16
NanoBytes wrote:
Ok, revolution, how would i round the entire number off?
Rounding is usually only done for display purposes. During internal calculations rounding is generally not done, and is usually not needed.
02 Feb 2012, 04:16
NanoBytes

Joined: 02 Jun 2011
Posts: 57
Location: Iowa, United States
NanoBytes 02 Feb 2012, 05:35
I know, I am working on a atof procedure
02 Feb 2012, 05:35
revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 19076
revolution 02 Feb 2012, 05:52
02 Feb 2012, 05:52
JohnFound

Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 02 Feb 2012, 06:46
Note that from mathematical point of view 12.199(9) = 12.2 and this is exact equation.
02 Feb 2012, 06:46
revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 19076
revolution 02 Feb 2012, 09:40
JohnFound wrote:
Note that from mathematical point of view 12.199(9) = 12.2 and this is exact equation.
The FPU cannot store enough precision for that.

The closest is 12.19999999999999929
The next value is 12.20000000000000107
02 Feb 2012, 09:40
smiddy

Joined: 31 Oct 2004
Posts: 557
smiddy 02 Feb 2012, 13:00
revolution wrote:
JohnFound wrote:
Note that from mathematical point of view 12.199(9) = 12.2 and this is exact equation.
The FPU cannot store enough precision for that.

The closest is 12.19999999999999929
The next value is 12.20000000000000107
That's interesting! What is the limiting factor causing the precision error?
02 Feb 2012, 13:00
JohnFound

Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 02 Feb 2012, 13:18
Quote:
That's interesting! What is the limiting factor causing the precision error?

The registers size of course.
02 Feb 2012, 13:18
revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 19076
revolution 02 Feb 2012, 13:33
smiddy wrote:
That's interesting! What is the limiting factor causing the precision error?
We can get more precision by using a larger mantissa size, but since the storage is binary and the output is decimal then there will always be some error. You can never completely remove the error, but you can make it arbitrarily small with an arbitrarily large mantissa.
02 Feb 2012, 13:33
Matrix

Joined: 04 Sep 2004
Posts: 1166
Location: Overflow
Matrix 02 Feb 2012, 21:38
revolution wrote:
smiddy wrote:
That's interesting! What is the limiting factor causing the precision error?
We can get more precision by using a larger mantissa size, but since the storage is binary and the output is decimal then there will always be some error. You can never completely remove the error, but you can make it arbitrarily small with an arbitrarily large mantissa.

or we just create a struct doubledouble and stack 2 double values in it, and get 2x precision bits
02 Feb 2012, 21:38
 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First

 Jump to: Select a forum Official----------------AssemblyPeripheria General----------------MainTutorials and ExamplesDOSWindowsLinuxUnixMenuetOS Specific----------------MacroinstructionsOS ConstructionIDE DevelopmentProjects and IdeasNon-x86 architecturesHigh Level LanguagesProgramming Language DesignCompiler Internals Other----------------FeedbackHeapTest Area

Forum Rules:
 You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot vote in polls in this forumYou cannot attach files in this forumYou can download files in this forum