How far after decimal point is accurate in fasmg?

flat assembler
Message board for the users of flat assembler.

Index > Main > How far after decimal point is accurate in fasmg?

Author

Thread

l4m2

Joined: 15 Jan 2015
Posts: 674

l4m2 02 Jan 2017, 01:26

Code:

y = 1.0 / 12345678
rept 80
  y = 10 * y
  n = 0
  while y>=1
    n = n+1
    y = y-1
  end while
  display '0'+n
end repeat

What I got is (0.)00000008100000664200054464404466081166218655665115538931309903877111288457712216
while it is actually 8100000664200054464404466081166218655629929761654240455647717363112823775(e-80)(calculated in machematica)
so how many digits can be guaranteed to be right?

02 Jan 2017, 01:26

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8376
Location: Kraków, Poland

Tomasz Grysztar 02 Jan 2017, 07:04

fasmg manual, 4. Expression values wrote:

The floating-point numbers are handled by the assembler in the binary form. Their range and precision are at least as high as they are in the longest floating-point format that the assembler is able to produce in the output.

Please also take a look at the other thread where I mention how you can adjust the maximum available precision when re-assembling fasmg.

02 Jan 2017, 07:04

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum