flat assembler
Message board for the users of flat assembler.

Index > Compiler Internals > eqtype. A bug or not a bug?

Goto page Previous  1, 2
Author
Thread Post new topic Reply to topic
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 13 Feb 2012, 09:20
Tomasz Grysztar
I think, there's a better solution, than adding a new boolean operator. Currently it's only allowed to compare expressions of the same type (relocatable with relocatable and non-relocatable with non-relocatable), and comparing a relocatable expression with a non-relocatable expression results in the error "invalid use of symbol":
Code:
format PE GUI 4.0

start:
ret

if start = 0
    display 'equals',13,10
else
    display 'differs',13,10
end if

section '.reloc' fixups readable discardable    


However if you make such comparison valid and returning always false for "=" and true for "<>" (I think, it's a reasonable modification), then it's possible to check, whether an expression is relocatable, without the additional operator with the following workaround:
Code:
format PE GUI 4.0

start:
ret

virtual
      dq start
    load start.noreloc qword from $$
end virtual

if start.noreloc = start
     display 'non-relocatable',13,10
else
   display 'relocatable',13,10
end if

section '.reloc' fixups readable discardable    

(btw., if you remember we had a discussion about documenting the compiler's internal word size. If you make it undocumented and try to enlarge in future versions, this approach becomes worthless)

And if you still want to add an additional operator, than a better idea would be to transform a relocatable value into a non-relocatable with an operator named "fixate" or "anchor" or "settle" (i.e. just remove the relocatability flag). Then the previous example would be simplified as follows:
Code:
format PE GUI 4.0

start:
ret

if fixate start = start
       display 'non-relocatable',13,10
else
   display 'relocatable',13,10
end if

section '.reloc' fixups readable discardable    


P.S. Btw., I use the trick with "virtual" to remove relocatability sometimes. Thus adding the fixating operator would be valueable.
Post 13 Feb 2012, 09:20
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8267
Location: Kraków, Poland
Tomasz Grysztar 13 Feb 2012, 11:35
l_inc wrote:
However if you make such comparison valid and returning always false for "=" and true for "<>" (I think, it's a reasonable modification)
For me it is not reasonable at all. I put much of effort to make fasm very reliable in code-resolving related areas, and the "=" operator has a very strong meaning. If you compare non-relocatable value with a simple number, it can become "true" of "false" at the runtime depending on how your code was relocated, so it is something like writing "if eax=0" and expecting assembler to give definite answer.

l_inc wrote:
And if you still want to add an additional operator, than a better idea would be to transform a relocatable value into a non-relocatable with an operator named "fixate" or "anchor" or "settle" (i.e. just remove the relocatability flag).
If you known what base the given value is relative to, you can just subtract the base address to get the fixed value (the RVA operator in PE format is a special case of this, it subtracts the known base value which you could have trouble defining as a label). And if you don't know what it is relative to, then such operator would not be very useful, because it would be equivalent to saying "subtract some value I don't know from this one". And you couldn't know what was subtracted and therefore what the result means.

The "virtual" trick may allow to retrieve the value that fasm generates into opcode, but the value itself is useless if you do not know exact way of how it should be relocated to become correct one. With relocatable PE it is not a problem, but in formats like ELF there are many different relocation types and you can even specify them to add an arbitrary value to the value from code before relocating, which makes retrieval of value from opcode a very risky business.
Post 13 Feb 2012, 11:35
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8267
Location: Kraków, Poland
Tomasz Grysztar 13 Feb 2012, 11:58
But your proposal did give me a hint for a better operator - it should be operator that would allow to test whether two values are relative to the same base (and so that substracting one from the other would be allowed and would give absolute value in result). Again, there is a problem with inventing the time, for now let's say it would be "samebaseas" or "relativeto" - then one wouldn't need the previously discussed hypothetical operator to check whether value is relocatable, it would be enough to do:
Code:
if mylabel relativeto 0    
Post 13 Feb 2012, 11:58
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 13 Feb 2012, 13:17
Tomasz Grysztar
Quote:
If you compare non-relocatable value with a simple number, it can become "true" of "false" at the runtime depending on how your code was relocated, so it is something like writing "if eax=0" and expecting assembler to give definite answer.

First of all, I assume, you mean "compare a relocatable value". Because under "relocatable" I understand, that the value can be relocated (changed) at the load time.
Secondly, let me explain my point about returning false. Currently in your implementation every numeric value is a tuple, which consists of the actual value and the flag "relocatable" (I'm not aware of ELF's abilities, probably it should be a tuple with the actual value and the currently computed base, if ELF's allow different code parts to be relocated asynchronously, i.e. with different deltas). A tuple is equal to another tuple if and only if all of their corresponding components are equal. Therefore it's absolutely logical to return inequality if the flag "relocatable" is not equal to the flag of the compared value.
Quote:
such operator would not be very useful, because it would be equivalent to saying "subtract some value I don't know from this one".

That's not true. I have my overloaded implementation of display, which allows to easily display numeric values. It makes use of the following macro (to post the complete implementation would be too much code):
Code:
;Displays a hexadecimal representation of a number
macro dispHex num*, padding, leader, trailer
{
 local digCount,dig,number
   
    number = num
        virtual
             dq number
           load number qword from $$
   end virtual
 if number < 0
            number = number + 100000000h
        end if
      digCount = 0
        while 1 shl (digCount*4) <= number
               digCount = digCount + 1
     end while
   if digCount = 0
             digCount = 1
        end if
      if ~ leader eq
              display leader
      end if
      if ~ padding eq
             if digCount < padding
                    times (padding-digCount) display '0'
              end if
      end if
      repeat digCount
             dig = number shr ((digCount-%)*4) and 0Fh+'0'
             if dig > '9'
                   dig = dig-'0'+('A'-10)
          end if
              display dig
 end repeat
  if ~ trailer eq
             display trailer
     end if
}    

As you can see I make use of double reassignment: first with = and second with dq->load. This allows to display values like "$" or "$-$$" independently of the context (with or without relocations enabled), and I find it very handy, even knowing, that the actual value of "$" can be different after the image is loaded.
Quote:
it would be enough to do:
Code:
if mylabel relativeto 0    

Could you explain the functionality? Because I don't understand, why the same expression
Code:
$ relativeto 0    
with relocations enabled and without them would be evaluated differently.

Another example of using this feature is the following:
Code:
;Allows to reduce code size by replacing <push const> with <push reg>
;usage:
;BeginRegIsConst ebx,0
;        mov eax,0
;EndRegIsConst
macro BeginRegIsConst reg*,const*
{
  local block_closed
  if ~ defined block_closed
           display 'Error: current block is not closed',13,10
            rb -1
       end if
      purge EndRegIsConst
 macro push [args]
   \{
            \common \local \isconst,\buf
                    \isconst = 0
                       if 0 eqtype args
                            virtual
                                     dq args
                                     load \buf qword from $$
                            end virtual
                         if \buf = const
                                    push reg
                                    \isconst = 1
                               end if
                      end if
                      if ~ \isconst
                              push args
                   end if
      \}
    macro mov [args]
    \{
            \common \local \isconst,\buf
                    \isconst = 0
                       match first=,second,args
                    \\{
                          if 0 eqtype second
                                  virtual
                                             dq second
                                           load \buf qword from $$
                                    end virtual
                                 if \buf = const
                                            mov first,reg
                                               \isconst = 1
                                       end if
                              end if
                      \\}
                  if ~ \isconst
                              mov args
                    end if
      \}
    macro pushd [args] \{ \common push args \}
     macro BeginRegIsConst dummy1*,dummy2*
       \{
            display 'Error: ',`reg,' still equals ',`const,13,10
                err
 \}
    macro EndRegIsConst \{ purge push,pushd,mov,BeginRegIsConst \}
  macro EndRegIsConst
 \{
            EndRegIsConst
               purge EndRegIsConst
         block_closed = 1
    \}
}    

I hope, it's clear from the comments, why this code is very handy. However in this case the reassignment with dq->load is exactly, what is not desired, because the second parameter of the macro could accidentally be equal to a some relocatable value and the program will become incorrect. Therefore instead of just clearing the relocatability flag I'd need a possibility to examine it, so that I do the replacement only for the fixed numeric values, that cannot be relocated at load time.
Post 13 Feb 2012, 13:17
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8267
Location: Kraków, Poland
Tomasz Grysztar 13 Feb 2012, 13:56
l_inc wrote:
First of all, I assume, you mean "compare a relocatable value". Because under "relocatable" I understand, that the value can be relocated (changed) at the load time.

Yes, that was a typo, sorry.

l_inc wrote:
Secondly, let me explain my point about returning false. Currently in your implementation every numeric value is a tuple, which consists of the actual value and the flag "relocatable"

Not, it is more complex than that...

l_inc wrote:
(I'm not aware of ELF's abilities, probably it should be a tuple with the actual value and the currently computed base, if ELF's allow different code parts to be relocated asynchronously, i.e. with different deltas)
...and the complexity I mentioned above is in part related to ELF/COFF abilities. PE format is not really representative when speaking about relocations.

l_inc wrote:
A tuple is equal to another tuple if and only if all of their corresponding components are equal.

This work only one way - if all components are equal, the equality of resulting values follows. But even when they are not equal, the result values could become the same.

l_inc wrote:
Therefore it's absolutely logical to return inequality if the flag "relocatable" is not equal to the flag of the compared value.

Not, it is not - it is like returning false when you try to compare "eax" with "0". It is not possible to determine result of such comparison at assembly time, it is a run-time construct.

l_inc wrote:
Quote:
such operator would not be very useful, because it would be equivalent to saying "subtract some value I don't know from this one".

That's not true. I have my overloaded implementation of display, which allows to easily display numeric values. (...)

This might be working out for you with relocatable PE, where there is only one possible base of relocations and thus you know what the base is (and, as I wrote in earlier post, if you know what the base is, you can just substract it). But even then when you display both absolute values and offset from base in the same way, on reading this output you may not be able to tell whether this is just some computed number (which will always be the same at run-time), or an address which can become something different at run-time.

l_inc wrote:
Quote:
it would be enough to do:
Code:
if mylabel relativeto 0    

Could you explain the functionality? Because I don't understand, why the same expression
Code:
$ relativeto 0    
with relocations enabled and without them would be evaluated differently.
The "$ relativeto 0" would be false in relocatable format, and it is related to the fact that you cannot do computations like "($-0) shr 3" in such case.

In general, it would work like this:
Code:
if mylabel relativeto 0

  dd mylabel xor 0xABCD ; this is just a number, I can do any computations on it

else if mylabel relativeto ebp

  offset_in_stack = mylabel - ebp

else if mylabel relativeto $

  jmp mylabel ; I know this jump will never generate relocation

else if mylabel relativeto some_external_array

  offset_in_array = mylabel - some_external_array

end if    
Post 13 Feb 2012, 13:56
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 19872
Location: In your JS exploiting you and your system
revolution 13 Feb 2012, 14:04
Tomasz Grysztar wrote:
Code:
else if mylabel relativeto ebp

  offset_in_stack = mylabel - ebp     
Just being able to do that alone would make the whole thing worthwhile IMO. I always wanted to be able to distinguish hidden base registers inside labels.
Post 13 Feb 2012, 14:04
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 13 Feb 2012, 14:30
Tomasz Grysztar
Quote:
But even when they are not equal, the result values could become the same.

I actually proposed to forget about such thing like "result values" and compute only on the basis of tuples (vectors). Then it would work both ways. And I don't see, why it would violate correctness. But if you say, the thing is more complicated in ELF's, I'll have to blindly believe it cannot be mapped to the arithmetic with vectors (may be the vector should just be extended with one or two components like "base register"?).
Quote:
it is like returning false when you try to compare "eax" with "0"

Well, yes. So what? The returned false is in this case the compile time comparison which does not have to reflect the run time. It has never been a problem for you to compare "eax" with "0" and return false when compared with eqtype. Why? Because it's just a different type of comparison. If you want a run time comparison, you always have cmp.
Quote:
on reading this output you may not be able to tell whether this is just some computed number (which will always be the same at run-time), or an address which can become something different at run-time

That's not a problem, as long as I'm aware of it. I just appreciate it's possible to remove the relocatability with such workaround.
Quote:
The "$ relativeto 0" would be false in relocatable format, and it is related to the fact that you cannot do computations like "($-0) shr 3" in such case

OK. But "relativeto" is IMHO very confusing here. I find "samebaseas" better. "relativeto" means for me a kind of offset. Like "rva" is an offset from the image base.
Quote:
Code:
else if mylabel relativeto $ 
  jmp mylabel ; I know this jump will never generate relocation    

Wouldn't you know the jump will never generate relocation without checking this? Direct near/short jumps never generate relocations. To generate a relocation it should be indirect or far.
Post 13 Feb 2012, 14:30
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8267
Location: Kraków, Poland
Tomasz Grysztar 13 Feb 2012, 15:27
l_inc wrote:
Well, yes. So what? The returned false is in this case the compile time comparison which does not have to reflect the run time. It has never been a problem for you to compare "eax" with "0" and return false when compared with eqtype. Why? Because it's just a different type of comparison. If you want a run time comparison, you always have cmp.
Yes, "eqtype" is a different kind of comparison, the lexical one. But "=" is numeric comparison that compares the computed values. If I allowed "eax = 0" evaluate to false, obvious mis-comprehensions would follow. fasm makes sure that values you assume to be equal are equal and there was much effort put into ensuring that in whole fasm's architectural design.

l_inc wrote:
Wouldn't you know the jump will never generate relocation without checking this? Direct near/short jumps never generate relocations. To generate a relocation it should be indirect or far.
In object format jump may generate relocation - when you jump to other section, or call the external procedure.
Post 13 Feb 2012, 15:27
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 13 Feb 2012, 16:20
Tomasz Grysztar
Quote:
fasm makes sure that values you assume to be equal are equal and there was much effort put into ensuring that in whole fasm's architectural design.

OK. That's not problem for me. Thank you for the explanation. Just, please, don't remove the trick with virtual. Because I think, with 129-bit internal arithmetic I will get wrong values, when trying to do dq->load (or let's say do(from define octaword)->load).
Quote:
In object format jump may generate relocation - when you jump to other section, or call the external procedure.

OK. Thank you for the information. I'm looking forward to be able to use the new directive. Smile
Post 13 Feb 2012, 16:20
View user's profile Send private message Reply with quote
shutdownall



Joined: 02 Apr 2010
Posts: 517
Location: Munich
shutdownall 13 Feb 2012, 16:33
Tomasz Grysztar wrote:
it would be enough to do:
Code:
if mylabel relativeto 0    


I like the keyword relativeto. Sounds a little bit italian like Iveco.
Spaghetti carbonara et una coca cola ... Cool
Post 13 Feb 2012, 16:33
View user's profile Send private message Send e-mail Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 13 Feb 2012, 16:39
Oh. For that reason there should be definitely added the directive 相同的基.
Post 13 Feb 2012, 16:39
View user's profile Send private message Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4624
Location: Argentina
LocoDelAssembly 13 Feb 2012, 16:51
LocoDelAssembly wrote:
Also, would this be the first time fasm introduces a keyword composed of more than one word?
No, as an example you have eqtype which has been repeated in this thread countless times. What's wrong with you?

Because of the paragraph above, I think relativeto, relto, basedon would be nice.
Post 13 Feb 2012, 16:51
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 19 Feb 2012, 13:41
I don't think my opinion is going to change something (because I have already justified it before), but I still think relativeto is a very bad choise. Aside from it's large length the name is not consistent with it's commutativity property. eqbase (analogously to the here discussed eqtype) would be IMHO a much better name.
Post 19 Feb 2012, 13:41
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8267
Location: Kraków, Poland
Tomasz Grysztar 19 Feb 2012, 13:56
I do not think that "relativeto" has a problem with implied non-commutativity, it is like saying two values are related to each other, they are each other's relatives. I preferred to avoid suggesting that there is some base value, perhaps sometimes there is no real base at all, only the relation of two values.
Post 19 Feb 2012, 13:56
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 19 Feb 2012, 15:47
Tomasz Grysztar
I thought about that explanation, and have no counterarguments. May be I'm just bound to a single understanding of an ambiguous expression. However it would be nice to see some native speakers opinions.
Post 19 Feb 2012, 15:47
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 21 Jun 2014, 12:13
Tomasz Grysztar
No native speakers seem to have been willing to express an opinion. Thus here's my last question on that.
Quote:
it is like saying two values are related to each other

Then why wasn't it a "relatedto"?

_________________
Faith is a superposition of knowledge and fallacy
Post 21 Jun 2014, 12:13
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2023, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.