flat assembler
Message board for the users of flat assembler.
![]() |
Author |
|
revolution
12.2 cannot be perfectly represented as a floating point number. The FPU will use the closest approximation available 12.199999... Generally one would round the numbers to the required precision for display and the small error in the internal representation would not be seen.
|
|||
![]() |
|
NanoBytes
Ahh, i think i figured it out
Code: MOV [Integer1],15 FILD [Integer1] MOV [Integer1],2 FILD [Integer1] MOV [Integer1],10 FIDIV [Integer1] FSUBP st1,st0 FSTP [Integer1] BTS [Integer1],2 FLD [Integer1] Though, i am worried, I am converting from the registers 10 bytes to the integers 4 bytes just to correctly add two numbers. Is there a more accurate way to do this? _________________ He is no fool who gives what he cannot keep to gain what he cannot loose. |
|||
![]() |
|
NanoBytes
Ok, revolution, how would i round the entire number off?
|
|||
![]() |
|
revolution
NanoBytes wrote: Ok, revolution, how would i round the entire number off? |
|||
![]() |
|
NanoBytes
I know, I am working on a atof procedure
|
|||
![]() |
|
revolution
|
|||
![]() |
|
JohnFound
Note that from mathematical point of view 12.199(9) = 12.2 and this is exact equation.
|
|||
![]() |
|
revolution
JohnFound wrote: Note that from mathematical point of view 12.199(9) = 12.2 and this is exact equation. The closest is 12.19999999999999929 The next value is 12.20000000000000107 |
|||
![]() |
|
smiddy
revolution wrote:
|
|||
![]() |
|
JohnFound
Quote: That's interesting! What is the limiting factor causing the precision error? The registers size of course. |
|||
![]() |
|
revolution
smiddy wrote: That's interesting! What is the limiting factor causing the precision error? |
|||
![]() |
|
Matrix
revolution wrote:
or we just create a struct doubledouble and stack 2 double values in it, and get 2x precision bits |
|||
![]() |
|
< Last Thread | Next Thread > |
Forum Rules:
|
Copyright © 1999-2020, Tomasz Grysztar. Also on GitHub, YouTube, Twitter.
Website powered by rwasa.