Floating point division problem

Index > Windows > Floating point division problem

Author

Thread

TmX

Joined: 02 Mar 2006
Posts: 825
Location: Jakarta, Indonesia

TmX

I'm trying to do floating point division: 48.6 / 17.1

Code:

format PE console 4.0
entry main
include 'win32a.inc'

section '.data' data readable writeable
num1 dq 48.6
num2 dq 17.1
result dq ?
fmt db "%g", 10
szBuff db 32 dup (0)

section '.code' code readable executable
main:
        fld qword [num1]
        fld qword [num2]
        fdivp 
        fstp qword [result]
        invoke printf, fmt, result 
        invoke ExitProcess, 0
        
        
section '.idata' import data readable
library kernel32,'kernel32.dll', msvcrt,'msvcrt.dll'
import kernel32, ExitProcess,'ExitProcess'
import msvcrt, printf, 'printf'

The expected result, obviously, is 2.8421...
Yet the output of the code is 4.21429e+259

So I checked the executable using OllyDbg

It seems that fistp gave the correct result, but it got messed up.
What is wrong here?

10 Jan 2013, 03:25

LocoDelAssembly
Your code has a bug

Joined: 06 May 2005
Posts: 4633
Location: Argentina

LocoDelAssembly

cinvoke printf, fmt, result, result+4 Question

10 Jan 2013, 04:02

TmX

Joined: 02 Mar 2006
Posts: 825
Location: Jakarta, Indonesia

TmX

LocoDelAssembly wrote:

cinvoke printf, fmt, result, result+4

Well, now the result is:
1.78705e-307

Anyway, I think you're right: C functions should be called with cinvoke, not invoke.

10 Jan 2013, 08:23

Athlon64

Joined: 28 Feb 2012
Posts: 5

Athlon64

cinvoke printf, fmt, dword [result], dword [result+4]

10 Jan 2013, 11:13

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 7845
Location: Kraków, Poland

Tomasz Grysztar

Code:

include 'win32ax.inc'

Code:

cinvoke printf, fmt, double [result]

10 Jan 2013, 12:21

TmX

Joined: 02 Mar 2006
Posts: 825
Location: Jakarta, Indonesia

TmX

Thanks Tomasz, works like a charm. Smile

BTW, why is double needed in the cinvoke part? Is C's double the equivalent of qword?

10 Jan 2013, 13:31

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 7845
Location: Kraków, Poland

Tomasz Grysztar

Yes, the "double" in C means double precision floating point value, which is a 64-bit value.

In case of fasm's macroinstructions "double" means simply that there will be double push instruction - that the size of value is twice the size of stack unit. And since in Win32 the size of stack unit is 32 bits, the size of doubled one is 64 bits.

10 Jan 2013, 14:00

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum