flat assembler
Message board for the users of flat assembler.

Index > Main > FPU: Rounding and Conversion

Author
Thread Post new topic Reply to topic
cod3b453



Joined: 25 Aug 2004
Posts: 619
cod3b453
[1] Form what I've read I can round up, down, nearest or truncate using control word and frndint. Is round up and down the same as ceiling and floor or would a ceiling/floor function have to add/subtract 0.5 and then round?

[2] Anyone know where to look or what to do to convert REAL10 to a string in decimal form and vice versa?

Many thanks!
Post 03 Dec 2006, 15:44
View user's profile Send private message Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
AMD64 Architecture Programmer’s Manual Volume 1: Application Programming wrote:
Table 6-2. Types of Rounding
RC Value Mode Type of Rounding
00
(default)
Round to nearest
The rounded result is the representable value closest
to the infinitely precise result. If equally close, the
even value (with least-significant bit 0) is taken.
01 Round down
The rounded result is closest to, but no greater than,
the infinitely precise result.
10 Round up The rounded result is closest to, but no less than, the
infinitely precise result.
11 Round toward
zero
The rounded result is closest to, but no greater in
absolute value than, the infinitely precise result.


You have two options, set the RC to the kind of operation you want to do or seting RC to round toward zero and add/subtract 0.5 before frndint

I have no info for you second question, sorry
Post 03 Dec 2006, 16:15
View user's profile Send private message Reply with quote
Jack



Joined: 16 Feb 2005
Posts: 21
Jack
what I did once was to parse a numeric string to form a string that represents the number in scientific format, then character by character convert the mantissa as a bcd integer, then use fbld to load the integer and then divide it by 10^exponent, you can do 18 digits that way if you want that extra digit you will have to do a little more work.
to go the other way, find the approximate exponent of 10 by multiplying the number by log10(2)
then multiply your number by 10^exponent, most of the time that works but sometimes you have to additionally multiply again by 10, once you have the integer store it with fbstp and convert the bcd number to string.
or you could just use Raymond's fpu lib http://www.ray.masmcode.com/
Post 04 Dec 2006, 01:13
View user's profile Send private message Reply with quote
cod3b453



Joined: 25 Aug 2004
Posts: 619
cod3b453
ok got ceiling and floor sorted, turns out RC = 00 works just adding / subracting 0.5.

I'm still looking through Raymond's fpu lib, but hopefully I'll get that working soon thanks guys!
Post 09 Dec 2006, 20:49
View user's profile Send private message Reply with quote
Goplat



Joined: 15 Sep 2006
Posts: 181
Goplat
cod3b453 wrote:
ok got ceiling and floor sorted, turns out RC = 00 works just adding / subracting 0.5.

That will give wrong answers on odd integers. 5 + 0.5 = 5.5 which rounds to 6. 5 - 0.5 = 4.5 which rounds to 4.
Post 09 Dec 2006, 21:21
View user's profile Send private message Reply with quote
cod3b453



Joined: 25 Aug 2004
Posts: 619
cod3b453
Goplat wrote:
cod3b453 wrote:
ok got ceiling and floor sorted, turns out RC = 00 works just adding / subracting 0.5.

That will give wrong answers on odd integers. 5 + 0.5 = 5.5 which rounds to 6. 5 - 0.5 = 4.5 which rounds to 4.

thanks Goplat i must have not tested odd integers Confused took me a while to work this out but I ended up add an integer check so that all integers are kept the same, other math tricks didn't seem to work.

Code:
tst dt 5.0
val dt 0.0
hlf dt 0.5

fld tword [tst] ; st0 = N
fld st0         ; st0 = N; st1 = N
frndint         ; st0 = int[N]; st1 = N
fsub st1,st0    ; st0 = int[N]; st1 = N - int[N]
fxch st1        ; st0 = N - int[N]; st1 = int[N]
fldz            ; st0 = 0; st1 = N - int[N]; st2 = int[N]
fcomip st0,st1  ; st0 = N - int[N]; st1 = int[N]
fwait
faddp st1,st0   ; st0 = N
jz @f

fld tword [hlf] ; st0 = ½; st1 = N
fsubp st1,st0   ; st0 = N - ½
frndint         ; st0 = int[N - ½]

@@:

fistp qword [val]
    


now that this seems to work properly i can use the log10(N) trick which uses the floor function for the exponent.
Post 13 Dec 2006, 23:10
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on GitHub, YouTube, Twitter.

Website powered by rwasa.