flat assembler
Message board for the users of flat assembler.

 Index > Main > FPU: Rounding and Conversion
Author
cod3b453

Joined: 25 Aug 2004
Posts: 618
cod3b453 03 Dec 2006, 15:44
[1] Form what I've read I can round up, down, nearest or truncate using control word and frndint. Is round up and down the same as ceiling and floor or would a ceiling/floor function have to add/subtract 0.5 and then round?

[2] Anyone know where to look or what to do to convert REAL10 to a string in decimal form and vice versa?

Many thanks!
03 Dec 2006, 15:44
LocoDelAssembly

Joined: 06 May 2005
Posts: 4624
Location: Argentina
LocoDelAssembly 03 Dec 2006, 16:15
AMD64 Architecture Programmer’s Manual Volume 1: Application Programming wrote:
Table 6-2. Types of Rounding
RC Value Mode Type of Rounding
00
(default)
Round to nearest
The rounded result is the representable value closest
to the infinitely precise result. If equally close, the
even value (with least-significant bit 0) is taken.
01 Round down
The rounded result is closest to, but no greater than,
the infinitely precise result.
10 Round up The rounded result is closest to, but no less than, the
infinitely precise result.
11 Round toward
zero
The rounded result is closest to, but no greater in
absolute value than, the infinitely precise result.

You have two options, set the RC to the kind of operation you want to do or seting RC to round toward zero and add/subtract 0.5 before frndint

I have no info for you second question, sorry
03 Dec 2006, 16:15
Jack

Joined: 16 Feb 2005
Posts: 21
Jack 04 Dec 2006, 01:13
what I did once was to parse a numeric string to form a string that represents the number in scientific format, then character by character convert the mantissa as a bcd integer, then use fbld to load the integer and then divide it by 10^exponent, you can do 18 digits that way if you want that extra digit you will have to do a little more work.
to go the other way, find the approximate exponent of 10 by multiplying the number by log10(2)
then multiply your number by 10^exponent, most of the time that works but sometimes you have to additionally multiply again by 10, once you have the integer store it with fbstp and convert the bcd number to string.
or you could just use Raymond's fpu lib http://www.ray.masmcode.com/
04 Dec 2006, 01:13
cod3b453

Joined: 25 Aug 2004
Posts: 618
cod3b453 09 Dec 2006, 20:49
ok got ceiling and floor sorted, turns out RC = 00 works just adding / subracting 0.5.

I'm still looking through Raymond's fpu lib, but hopefully I'll get that working soon thanks guys!
09 Dec 2006, 20:49
Goplat

Joined: 15 Sep 2006
Posts: 181
Goplat 09 Dec 2006, 21:21
cod3b453 wrote:
ok got ceiling and floor sorted, turns out RC = 00 works just adding / subracting 0.5.

That will give wrong answers on odd integers. 5 + 0.5 = 5.5 which rounds to 6. 5 - 0.5 = 4.5 which rounds to 4.
09 Dec 2006, 21:21
cod3b453

Joined: 25 Aug 2004
Posts: 618
cod3b453 13 Dec 2006, 23:10
Goplat wrote:
cod3b453 wrote:
ok got ceiling and floor sorted, turns out RC = 00 works just adding / subracting 0.5.

That will give wrong answers on odd integers. 5 + 0.5 = 5.5 which rounds to 6. 5 - 0.5 = 4.5 which rounds to 4.

thanks Goplat i must have not tested odd integers took me a while to work this out but I ended up add an integer check so that all integers are kept the same, other math tricks didn't seem to work.

Code:
```tst dt 5.0
val dt 0.0
hlf dt 0.5

fld tword [tst] ; st0 = N
fld st0         ; st0 = N; st1 = N
frndint         ; st0 = int[N]; st1 = N
fsub st1,st0    ; st0 = int[N]; st1 = N - int[N]
fxch st1        ; st0 = N - int[N]; st1 = int[N]
fldz            ; st0 = 0; st1 = N - int[N]; st2 = int[N]
fcomip st0,st1  ; st0 = N - int[N]; st1 = int[N]
fwait
faddp st1,st0   ; st0 = N
jz @f

fld tword [hlf] ; st0 = ½; st1 = N
fsubp st1,st0   ; st0 = N - ½
frndint         ; st0 = int[N - ½]

@@:

fistp qword [val]
```

now that this seems to work properly i can use the log10(N) trick which uses the floor function for the exponent.
13 Dec 2006, 23:10
 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First

 Jump to: Select a forum Official----------------AssemblyPeripheria General----------------MainTutorials and ExamplesDOSWindowsLinuxUnixMenuetOS Specific----------------MacroinstructionsOS ConstructionIDE DevelopmentProjects and IdeasNon-x86 architecturesHigh Level LanguagesProgramming Language DesignCompiler Internals Other----------------FeedbackHeapTest Area

Forum Rules:
 You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot vote in polls in this forumYou cannot attach files in this forumYou can download files in this forum