flat assembler
Message board for the users of flat assembler.
 Home   FAQ   Search   Register 
 Profile   Log in to check your private messages   Log in 
flat assembler > Main > How far after decimal point is accurate in fasmg?

Author
Thread Post new topic Reply to topic
l4m2



Joined: 15 Jan 2015
Posts: 576
How far after decimal point is accurate in fasmg?

Code:
y = 1.0 / 12345678
rept 80
  y = 10 * y
  n = 0
  while y>=1
    n = n+1
    y = y-1
  end while
  display '0'+n
end repeat


What I got is (0.)00000008100000664200054464404466081166218655665115538931309903877111288457712216
while it is actually 8100000664200054464404466081166218655629929761654240455647717363112823775(e-80)(calculated in machematica)
so how many digits can be guaranteed to be right?
Post 02 Jan 2017, 01:26
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6386
Location: Kraków, Poland

fasmg manual, 4. Expression values wrote:
The floating-point numbers are handled by the assembler in the binary form. Their range and precision are at least as high as they are in the longest floating-point format that the assembler is able to produce in the output.

Please also take a look at the other thread where I mention how you can adjust the maximum available precision when re-assembling fasmg.
Post 02 Jan 2017, 07:04
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >

Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2005 phpBB Group.

Main index   Download   Documentation   Examples   Message board
Copyright © 2004-2016, Tomasz Grysztar.