flat assembler
Message board for the users of flat assembler.

Index > Main > Bug in calculation with big numbers

Author
Thread Post new topic Reply to topic
SPTH



Joined: 24 Jul 2004
Posts: 91
SPTH
This macro shows the decimal representation of the input. See the output for big numbers.
Code:
macro displayDecNum num* ; useful for debugging
{
    dN1=  num MOD 10
    dN2= ((num-dN1)/10) MOD 10
    dN3= ((num-dN1-dN2)/100) MOD 10
    dN4= ((num-dN1-dN2-dN3)/1'000) MOD 10
    dN5= ((num-dN1-dN2-dN3-dN4)/10'000) MOD 10
    dN6= ((num-dN1-dN2-dN3-dN4-dN5)/100'000) MOD 10
    dN7= ((num-dN1-dN2-dN3-dN4-dN5-dN6)/1'000'000) MOD 10
    dN8= ((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7)/10'000'000) MOD 10
    dN9= ((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7-dN8)/100'000'000) MOD 10
    dN10=((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7-dN8-dN9)/1'000'000'000) MOD 10
    dN11=((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7-dN8-dN9-dN10)/10'000'000'000) MOD 10
    dN12=((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7-dN8-dN9-dN10-dN11)/100'000'000'000) MOD 10
    dN13=((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7-dN8-dN9-dN10-dN11-dN12)/1'000'000'000'000) MOD 10
    dN14=((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7-dN8-dN9-dN10-dN11-dN12-dN13)/10'000'000'000'000) MOD 10
    dN15=((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7-dN8-dN9-dN10-dN11-dN12-dN13-dN14)/100'000'000'000'000) MOD 10
    dN16=((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7-dN8-dN9-dN10-dN11-dN12-dN13-dN14-dN15)/1'000'000'000'000'000) MOD 10
    dN17=((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7-dN8-dN9-dN10-dN11-dN12-dN13-dN14-dN15-dN16)/10'000'000'000'000'000) MOD 10
    dN18=((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7-dN8-dN9-dN10-dN11-dN12-dN13-dN14-dN15-dN16-dN17)/100'000'000'000'000'000) MOD 10
    dN19=((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7-dN8-dN9-dN10-dN11-dN12-dN13-dN14-dN15-dN16-dN17-dN18)/1'000'000'000'000'000'000) MOD 10
    dN20=((num-dN1-dN2-dN3-dN4-dN5-dN6-dN7-dN8-dN9-dN10-dN11-dN12-dN13-dN14-dN15-dN16-dN17-dN18-dN19)/10'000'000'000'000'000'000) MOD 10

    display dN20+'0',dN19+'0',dN18+'0',dN17+'0',dN16+'0',dN15+'0',dN14+'0',dN13+'0',dN12+'0',dN11+'0',dN10+'0',dN9+'0',dN8+'0',dN7+'0',dN6+'0',dN5+'0',dN4+'0',dN3+'0',dN2+'0',dN1+'0',13,10
}

displayDecNum (0x1FFF'FFFF'FFFF'FFFF) ; 02305843009213693951
displayDecNum (0x5FFF'FFFF'FFFF'FFFF) ; 06917529027641081855
displayDecNum (0x9FFF'FFFF'FFFF'FFFF) ; 0691752902)*,/0(/(+)  <- !!!
displayDecNum (0xAFFF'FFFF'FFFF'FFFF) ; 0576460752-0-,.-,((/  <- !!!
displayDecNum (0xFFFF'FFFF'FFFF'FFFF) ; 0000000000000000000/  <- !!!
    


Seems to be a FASM bug?
Post 29 Apr 2011, 04:24
View user's profile Send private message Reply with quote
typedef



Joined: 25 Jul 2010
Posts: 2913
Location: 0x77760000
typedef
I get
02305843009213693951
06917529027641081855
0691752902)*,/0(/(+)
0576460752-0-,.-,((/
0000000000000000000/


Where is the bug ?
is it here == 0xFF ?
Post 29 Apr 2011, 04:44
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17463
Location: In your JS exploiting you and your system
revolution
I use this for displaying decimal numbers:
Code:
macro display_decimal value {
    local leading_zero,digit,divisor,number
    number=value
    if number=1 shl 63
    display '-9223372036854775808'
    else
        if number<0
          number=-number
      display '-'
   end if
      leading_zero=0
      divisor=1000000000000000000
 while divisor>0
      digit=number/divisor
        leading_zero=leading_zero+digit
     if leading_zero | (divisor=1)
           display digit+'0'
         number=number-digit*divisor
     end if
      divisor=divisor/10
      end while
    end if
}
macro display_unsigned value {
    local number
    number=value
    if number<0
      number=number-10000000000000000000
  if number<0
          number=number+1000000000000000000
           display '9'
   else
            display '1'
   end if
    end if
    display_decimal number
}    
SPTH: The problem with your conversion is that you didn't handle negative numbers properly.
Post 29 Apr 2011, 04:55
View user's profile Send private message Visit poster's website Reply with quote
SPTH



Joined: 24 Jul 2004
Posts: 91
SPTH
Ahh, I didnt know that FASM variables use the highest bit as sign.

Thanks for the fast answeres!
Post 29 Apr 2011, 05:14
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on YouTube, Twitter.

Website powered by rwasa.