flat assembler
Message board for the users of flat assembler.

Index > Compiler Internals > db 0. A bug or not a bug?

Goto page Previous  1, 2, 3, 4  Next
Author
Thread Post new topic Reply to topic
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 28 Feb 2012, 00:45
Tomasz Grysztar wrote:
OK, after all I managed to overcome my concerns and I started working on it (I mean adding the additional sign bit to internal calculations).
This is a good thing. My previous thoughts can finally become a reality.
Post 28 Feb 2012, 00:45
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 28 Feb 2012, 00:51
Tomasz Grysztar
Do I understand it right, that after this improvement there will be no way to store-load a value and obtain the original result?
E.g. the following code
Code:
rb 100
store qword -1 at 0
load x qword from 0
if x = -1
    display 'Consistent store-load',0
else
 display 'Inconsistent store-load',0
end if
    

will display "Inconsistent store-load".
Post 28 Feb 2012, 00:51
View user's profile Send private message Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4624
Location: Argentina
LocoDelAssembly 28 Feb 2012, 01:14
l_inc, If you replace all the occurrences of qword with dword you'll get 'Inconsistent store-load' message too even with the current fasm version. Perhaps it is time to introduce yet another feature like "load signed ..."?
Post 28 Feb 2012, 01:14
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 28 Feb 2012, 01:23
Code:
macro signExtend.dword v.out, v.in {
    v.out = v.in or ((v.in and (1 shl 31)) * (((1 shl 32)-1) shl 1))
}    
Extra brackets are just for clarity
Post 28 Feb 2012, 01:23
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 28 Feb 2012, 01:59
LocoDelAssembly
Quote:
l_inc, If you replace all the occurrences of qword with dword you'll get 'Inconsistent store-load' message too even with the current fasm version.

I know. That's why I explicitly written: "There will be no way...". There is a way now: to use qword and dq.
Quote:
Perhaps it is time to introduce yet another feature like "load signed ..."?

Not that simple. After a value is stored with store or dq, it's sign would be irrevocably lost. And "signed" load would not be able to decide, what sign should be loaded.
revolution
I'm not sure, who's post you're answering. But if mine:
First, I suppose, this won't work with qword values because of the overflow.
And second, the purpose is to get a guaranteed same value as was stored in current addressing space, not to sign extend a value (if stored 0xFFFFFFFF, I want to get the same value back, not the -1).
Post 28 Feb 2012, 01:59
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 28 Feb 2012, 02:08
l_inc: You are trying to squeeze a 65-bit value into a 64-bit hole. You will always lose some information that way. If you really need to keep all the info then you have to make space for an extra bit somewhere and store the sign there. You can't get something for nothing.
Code:
x = -1
store qword x at 0 ;we lost the extra sign bit, the hole was not big enough to take the full number
store byte (x shr 64) at 8 ;if we need the sign bit then we gotta store it somewhere!    
Post 28 Feb 2012, 02:08
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 28 Feb 2012, 02:17
revolution
Quote:
You are trying to squeeze a 65-bit value into a 64-bit hole.

Yes. In this way I try to show, why I don't like the improvement.
Quote:
Code:
store byte (x shr 64) at 8 ;if we need the sign bit then we gotta store it somewhere!    

I assume, this is exactly, what is not going to be possible. I can live with that solution (even though I will need to adjust a significant part of my macros, so that they work correctly after the improvement), but my prognose is, that the old simple and working
Code:
dq x
load y qword from $-8    

will bloat into 8-16 lines with ugly if-else-end if's.
Post 28 Feb 2012, 02:17
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 28 Feb 2012, 02:23
This will only affect you if you do some sort of arithmetic that requires the current particular sign handling i.e. if you use division or multiplication on the values loaded back from the memory store. I don't see how simple data movement will be affected.
Post 28 Feb 2012, 02:23
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 28 Feb 2012, 02:29
revolution
An example of an affected macro:
Code:
;Displays a decimal representation of a number
macro dispDec num*, padding, leader, trailer
{
      local digCount,tenPow,number

    number = num
        virtual
             dq number
           load number qword from $$
   end virtual
 if number < 0
            display '-'
               number = -number
    end if
      digCount = 0
        tenPow = 1
  while tenPow <= number
           tenPow = tenPow*10
          digCount = digCount + 1
     end while
   if digCount = 0
             digCount = 1
                tenPow = 10
 end if
      if ~ leader eq
              display leader
      end if
      if ~ padding eq
             if digCount < padding
                    times (padding-digCount) display '0'
              end if
      end if
      repeat digCount
             tenPow = tenPow/10
          display number/tenPow + '0'
               number = number mod tenPow
  end repeat
  if ~ trailer eq
             display trailer
     end if
}    
Post 28 Feb 2012, 02:29
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 28 Feb 2012, 08:06
LOAD and STORE are just a special data directives that load or store some amount of bytes, just like DQ. The way you use them here is equal to displaying the content of bytes generated by DQ directive, and you already loose some information, like relocation (if the value is relocatable then such display is not very useful, since you no longer know what is it relative to). If you hoped that this would be a kind of complete storage of numerical constant, then you misinterpreted the purpose of those directives. You just display what bytes fasm generates when asked to write the value as 8-byte one.

Adding the 65-th bit is going to clean up some quirks in which DQ behaved differently from DD/DW/DB - thus for any calculations that fit in 64 bits it is going to get transparent what format fasm uses internally (that is: DQ will too become compatible with what it would be like if fasm used 129-bit, or even arbitrary 2-adic precision internally).

It may break some things if someone misused the old QWORD bugs. However the above macro will still do exactly what it does now, it will display the value of 8 bytes generated with DQ directive.
Post 28 Feb 2012, 08:06
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 28 Feb 2012, 10:11
Tomasz Grysztar
Quote:
Adding the 65-th bit is going to clean up some quirks in which DQ behaved differently from DD/DW/DB

I know what the improvement is intended to do. IMHO it breaks too many things.
Quote:
It may break some things if someone misused the old QWORD bugs.

There has been never provided a definition for use and misuse. Especially because (as you also said) there were found (and became standard) interesting usage examples for directives never intended for those purposes.
Quote:
However the above macro will still do exactly what it does now, it will display the value of 8 bytes generated with DQ directive.

Doing the same means to return same result for any usage. It won't return the same result for dispDec -1, will it?
Quote:
and you already loose some information, like relocation (if the value is relocatable then such display is not very useful, since you no longer know what is it relative to)

We already discussed, that this behaviour is exactly the purpose, but forget this macro: I wouldn't like to repeat the same discussion again. Consider the following macro:
Code:
;allows to define a sorted array of signed dwords
macro ddSorted [vals*]
{
    common local count,qw1,qw2
          count = 0
           virtual
     forward
                     dq vals
                     count = count + 1
   common
                      repeat count-1
                              repeat count-%
                                      load qw1 qword from $$+(%-1)*8
                                      load qw2 qword from $$+%*8
                                  if qw1 > qw2
                                             store qword qw2 at $$+(%-1)*8
                                               store qword qw1 at $$+%*8
                                   end if
                              end repeat
                  end repeat
                  count = 0
   forward local newvals
                       load newvals qword from $$+count*8
                  count = count + 1
   common
              end virtual
 forward
             dd newvals
}    

Will it give the same result for the code ddSorted -2,2,0,+1,-1 as now? Or is the compilation going to fail? Any ideas of how to achieve the same with the new fasm behaviour?
Post 28 Feb 2012, 10:11
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 28 Feb 2012, 10:18
There is a standard technique that can be used to sort signed values with an unsigned comparison.
Code:
v1 ;value1
v2 ;value2
;signed compare
  if v1 > v2 ...
;unsigned compare
  if (v1 xor (1 shl 63)) > (v2 xor (1 shl 63)) ...    
Extra brackets used for clarity
Post 28 Feb 2012, 10:18
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 28 Feb 2012, 10:24
revolution
I assume, you didn't understand the operation. The sign is not defined by the MSB. It's lost by storing values. And there is no efficient way to sort an array under fasm without storing values.

Btw. the provided macro need to perform the same way with qwords after changing the last dd with dq.
Post 28 Feb 2012, 10:24
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 28 Feb 2012, 10:27
l_inc wrote:
There has been never provided a definition for use and misuse. Especially because (as you also said) there were found (and became standard) interesting usage examples for directives never intended for those purposes.
Well, your macro is a quite standard usage of LOAD/STORE directives, it does exactly what it is supposed to do - displays the 64-bit value generated by fasm's DQ directive into output.

l_inc wrote:
Doing the same means to return same result for any usage. It won't return the same result for dispDec -1, will it?
You just display a value of 64 bits of data, whether you choose to present it a signed 64-bit value, or an unsigned one, is your choice. It does not matter whether this 64-bit value was generated by writing "-1" or "0FFFFFFFFFFFFFFFFh", or "$-something" - the result produced by DQ is 64 bits of some data, nothing more.

l_inc wrote:
Will it give the same result for the code ddSorted -2,2,0,+1,-1 as now? Or is the compilation going to fail? Any ideas of how to achieve the same with the new fasm behaviour?
If you need to compare values as signed, you may need something like "load signed" - exactly what LocoDelAssembly proposed.
Post 28 Feb 2012, 10:27
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 28 Feb 2012, 10:33
l_inc wrote:
revolution
I assume, you didn't understand the operation. The sign is not defined by the MSB.
I don't understand why not.
Code:
;65-bit
-2 = 0x1fff...ffe 
-1 = 0x1fff...fff
 0 = 0x0000...000
+1 = 0x0000...001
+2 = 0x0000...002

;64-bit (like now)
-2 = 0x0fff...ffe 
-1 = 0x0fff...fff
 0 = 0x0000...000
+1 = 0x0000...001
+2 = 0x0000...002    
In both cases the MSb (bit 63 for a dq store) indicates the sign.

It is only different when you try to overflow the storage.
Code:
dq -(1 shl 63+1) ;What will happen? It should give an error    
Post 28 Feb 2012, 10:33
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 28 Feb 2012, 10:38
These problems arise from the fact that undocumented behaviors were used. That's another reason why I want to do this change - I want to have a consistent behavior independent of internal precision, so the values that fasm generates will be the same that infinite precision fasm would generate and the only difference from infinite precision fasm will be that it will fail with "out of range" error due to limitations of the internal representation.

After the corrections fasm will detect overflow not only on multiplication, but also on sum and substraction, and even in case of "shl", as fasm's "shl" operator was designed to not truncate values ("db 7 shl 8" does not generate truncated value, it overflows - and so should "dq 7 shl 62"). revolution, I'm sorry, your macro will probably not work because of this.
Post 28 Feb 2012, 10:38
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 28 Feb 2012, 10:59
revolution wrote:
Code:
dq -(1 shl 63+1) ;What will happen? It should give an error    
Yes, this already gives an "out of range" error with my current development version (and "dq -(1 shl 63)" assembles without a complaint).
Post 28 Feb 2012, 10:59
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 28 Feb 2012, 11:16
Tomasz Grysztar wrote:
revolution, I'm sorry, your macro will probably not work because of this.
Oh, okay. Are you talking about the signExtend.dword macro? But why? I never exceed the 64-bit value. It remains a 100% positive value between 0 and 2^64-1. I was not trying to generate a negative value, only to generate the equivalent 64-bit encoding of sign extension for 32-bit.
Post 28 Feb 2012, 11:16
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 28 Feb 2012, 11:32
Oh, sorry, I did not look at it in detail, I thought that you are trying to generate a sign bit with shifting. Something like:
Code:
macro signExtend.dword v.out, v.in {
    if ~ v.in and 80000000h
      v.out = v.in
    else
      v.out = v.in and 7FFFFFFFh - 80000000h
    end if
}    
The "qword" version of this macro will work correctly with the new fasm, too.
Post 28 Feb 2012, 11:32
View user's profile Send private message Visit poster's website Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4624
Location: Argentina
LocoDelAssembly 28 Feb 2012, 15:42
Would this if-less version work and be compatible with any future extra precision?
Code:
macro def [name, size]
{
   macro signExtend.#name v.out, v.in
   \{
      v.out = v.in or (0 - v.in shr (size - 1)) shl (size - 1)
   \}
}

def byte,  8,\
    word, 16,\
   dword, 32,\
   qword, 64

purge def    
(I think it is compatible with current fasm too)
But still, I think something like "load signed" should be native, and also store and data definition directives should support both signed and unsigned for easy range checking (if nothing is specified then the default is used which is the union of both if I understand the current behavior correctly)
Post 28 Feb 2012, 15:42
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2, 3, 4  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.