flat assembler
Message board for the users of flat assembler.

Index > Compiler Internals > 'err' false positive

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
BAiC



Joined: 22 Mar 2011
Posts: 272
Location: California
BAiC
As the subject says; it appears the err directive is allowed to be triggered during one of the earlier passes.
Post 13 Mar 2012, 10:56
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17278
Location: In your JS exploiting you and your system
revolution
Show your code please. We have no idea what you have done.
Post 13 Mar 2012, 10:57
View user's profile Send private message Visit poster's website Reply with quote
BAiC



Joined: 22 Mar 2011
Posts: 272
Location: California
BAiC
without the error
Image
with the error:
Image

The error is in the newer version as well (you can see the version in the picture) however there is an unrelated error in the new version that prevents me from using it.
Post 13 Mar 2012, 11:12
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7724
Location: Kraków, Poland
Tomasz Grysztar
manual, section 2.2.6 wrote:
The err directive, usually used to stop the assembly when some condition is met, stops the assembly immediately, regardless of whether the current pass is final or intermediate. So even when the condition that caused this directive to be interpreted is temporary, and would eventually disappear in the later passes, the assembly is stopped anyway.

The assert directive signalizes the error only if its expression is false after all the symbols have been resolved. You can use assert 0 in place of err when you do not want to have assembly stopped during the intermediate passes.
Post 13 Mar 2012, 11:13
View user's profile Send private message Visit poster's website Reply with quote
BAiC



Joined: 22 Mar 2011
Posts: 272
Location: California
BAiC
Assert is newer than 1.69.31
Post 13 Mar 2012, 11:19
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7724
Location: Kraków, Poland
Tomasz Grysztar
If look look at the same section in the manual that came with older versions, the "rb -1" idiom was recommended for that purpose. It is now deprecated though, since the "assert" has been introduced.
Post 13 Mar 2012, 11:20
View user's profile Send private message Visit poster's website Reply with quote
BAiC



Joined: 22 Mar 2011
Posts: 272
Location: California
BAiC
what's the point of having err at all if you can't determine whether it'll be triggered?
Post 13 Mar 2012, 11:22
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7724
Location: Kraków, Poland
Tomasz Grysztar
It was intended for fatal errors like the bad arguments for macro. It is the kind of error that means there is something wrong with the source itself. Also, the old version of fasm, which did not recognize "err" directive, did just signalize "illegal instruction" on it (which is also the "fatal" kind of error, which stops assembly immediately) - so when "err" was introduced, it just got a special error message, but still behaves like the illegal instruction.
Post 13 Mar 2012, 11:24
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
But there is (at least) one special case where err could lead to a false positive. This issue is however related to a late integer overflow detection and not to the err directive itself.
Code:
x = -1 shl 63
x = x+x
x = -x

if x = 0
   display "x is zero",13,10
 err
end if
    
Post 14 Mar 2012, 23:29
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17278
Location: In your JS exploiting you and your system
revolution
l_inc wrote:
Code:
x = -1 shl 63
x = x+x
x = -x

if x = 0
 display "x is zero",13,10
 err
end if
    
The same thing is seen here also:
Code:
if x = 0
        display "x is zero",13,10
        err
end if
x=1    
Post 15 Mar 2012, 02:33
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7724
Location: Kraków, Poland
Tomasz Grysztar
There is nothing really wrong with the "false positives" as you call them, as this is exactly what is documented. The ERR directive is truly like an illegal instruction, assembler gives up as soon as it stumbles upon it, for whatever reason. As I mentioned above, when assembler meets ERR directive it means that the source is broken, and the assembly should not proceed. The purpose of ERR directive is to signalize syntax error caused by the wrong usage of macros, etc. In general, it should better not be used to detect a conditions that are not obvious and permanent ones - that is the conditions that are not related directly to the source structure itself, but depend on some assembly-time values that may be changing or may need to be resolved.

If it was the "assert x<>0" that failed for you even though x was not zero, then it would really be false positive (that is: a bug, for that matter).

I think I'm going to change the ERR directive message to something like "error directive encountered in source file" instead of "error directive invoked in source file", which might have been more misleading.
Post 15 Mar 2012, 06:21
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7724
Location: Kraków, Poland
Tomasz Grysztar
l_inc: I later noticed that your sample is a bit more subtle - there is perhaps a wrong choice of the primary error to display from the ones assembler got. This is something I may want to correct a little.
Post 15 Mar 2012, 06:54
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17278
Location: In your JS exploiting you and your system
revolution
I'm not suggesting that this is a needed change but these "false positives" could be "fixed" if each variable has a flag to say the value is unknown.

For l_inc's example when the calculation overflows then the unknown flag can be set.

And for my follow up example each variable that has not yet been assigned a value would start in the unknown state.

Then whenever a comparison is performed it would never be able to match to any numeric value.

This unknown flag could also propagate through computations to the destination variable. I suppose this would work in such a way as the FPU NaN values. And perhaps this would allow a new comparison operator to check for unknown variable values.

Something to consider.
Post 15 Mar 2012, 09:49
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7724
Location: Kraków, Poland
Tomasz Grysztar
revolution wrote:
Then whenever a comparison is performed it would never be able to match to any numeric value.
This would not really help, because you could negate the logical expression and/or put ERR into "else" block, or do something more complex, and assembler wouldn't really know which path is better to choose when the value is not known yet.

Also, allowing to check the stage of resolving of some symbol would be against fasm's design - the expressions that you write in code should be the "equations" that are to be resolved, and thus it should not be allowed for them to contain any terms that would be related to resolving process itself, as that could break the whole thing.
Post 15 Mar 2012, 09:53
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
revolution
Quote:
The same thing is seen here also

That's not the same thing. Undefined values could imply undefined behaviour. But in my case the value of x is clearly defined (even though the assembler is unable to calculate it). To be more clear, I propose you to consider the simplified example:
Code:
x = 1
if x = 0
   err
end if    

There are three possible specifications to define, what can happen:
1) The err directive must not trigger an error, because x at no pass may become 0 at that point. That's the current specification of fasm. IMHO also the best case.
2) The err directive must trigger an error, because it's just present in the source after preprocessing. That's probably the Tomasz Grysztar's usage intention for the err directive.
3) The result is undefined by the specification. Thus whether err triggers an error, is specified to be implementation dependent. This is a valid specification, but is also very undesired, because it prevents a programmer from using err in many situations even though it would work as intended with current fasm implementation. This is the current behaviour of fasm.

Therefore following the current fasm specification the following example must not trigger an error as a result of err directive presence, because the source clearly defines x to be unequal to 0 at any pass.
Code:
x = (-1) shl 64
x = -x

if x = 0 
        err
end if    

But current fasm implementation follows the third specification case, as it triggers an error in this code, but does not trigger in the previous one.

Tomasz Grysztar
I posted that example, because I don't understand why overflow processing differs in the following cases:
Code:
;First case
x = (-1) shl 64
x = x shl 1

if x = 0 
    err 
end if    

Code:
;Second case
x = (-1) shl 64
x = -x

if x = 0 
        err
end if    

IMHO the error messages should be the same and point to the overflow line and no display output placed after the error line may appear, which is only true for the first case overflow.
Quote:
If it was the "assert x<>0" that failed for you even though x was not zero, then it would really be false positive (that is: a bug, for that matter).

Wouldn't be the following code one of such cases?
Code:
rb x*2
x = $-1    

The compilation fails because -1 is assigned to x, but that's not a valid solution. So the real failure reason is that fasm can't find the correct solution ("code cannot be generated") rather than invalid value for the rb directive.
Post 15 Mar 2012, 14:13
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7724
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
2) The err directive must trigger an error, because it's just present in the source after preprocessing. That's probably the Tomasz Grysztar's usage intention for the err directive.
No, not really - because then the error would be signalized by parser module, not assembler. There are some constructions like IF directive with EQ operator, which allow to hide some code from assembler, and the intention was that it should be possible to use ERR with such checks.

l_inc wrote:
IMHO the error messages should be the same and point to the overflow line and no display output placed after the error line may appear, which is only true for the first case overflow.
As manual states: "Only the messages created with the display directive during the last performed pass get actually displayed. In case when the assembly has been stopped due to an error, these messages may reflect the predicted values that are not yet resolved correctly."
The prediction mechanism used by fasm may vary from version to version. Assembler may have been trying to proceed with wrong predictions because it hoped the things could actually resolve themselves later, but then it sees that it doesn't lead anywhere, so ends with an error and a broken display output (which may nonetheless be sometimes useful and that's why it is provided - it may for example hint what (mis)predictions assembler made that resulted in its failure to assemble ultimately). So whether in some cases you get the display and in some not, may vary wildly with various versions of fasm - and it is documented that this is not a reliable info (though it may give some hint about into what dead end did the given version go in the resolving process).

l_inc wrote:
Quote:
If it was the "assert x<>0" that failed for you even though x was not zero, then it would really be false positive (that is: a bug, for that matter).

Wouldn't be the following code one of such cases?
Code:
rb x*2
x = $-1    

The compilation fails because -1 is assigned to x, but that's not a valid solution. So the real failure reason is that fasm can't find the correct solution ("code cannot be generated") rather than invalid value for the rb directive.
Yes, failing to find a correct solution may manifest with almost any other error message - when assembler goes into a dead end, the incorrectness of generated solution may have various different forms. But otherwise fasm would just have to display "code cannot be generated" for any error that occurs, because that's what happens - it cannot find the right solution because of some error.
Post 15 Mar 2012, 14:40
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
There are some constructions like IF directive with EQ operator, which allow to hide some code from assembler, and the intention was that it should be possible to use ERR with such checks.

OK. But it's still not clearly specified, when err should be expected to trigger an error. What should I expect compiling the following code? A clean compilation or undefined behaviour?
Code:
x = 1
if x = 0
   err
end if    

Quote:
So whether in some cases you get the display and in some not, may vary wildly with various versions of fasm and - and it is documented that this is not a reliable info

OK. That's convincing, but I still don't understand, why the behaviour should differ between the provided overflow cases.
Quote:
But otherwise fasm would just have to display "code cannot be generated" for any error that occurs, because that's what happens - it cannot find the right solution because of some error.

Yes. But attempts to be more specific should not lead to an obviously incorrect error message. "error: invalid value." is wrong, because the following code does provide a valid value of x = 1, that for some reason cannot be calculated by fasm.
Code:
rb x*2
x = $-1    

As long as fasm didn't get a consistent value for x at the second pass it should continue with more passes. If it fails to continue, then the specific reason for that is that "a consistent value for x could not be found within n passes", if you prefer to give an exact error message.
Post 15 Mar 2012, 15:13
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7724
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
What should I expect compiling the following code? A clean compilation or undefined behaviour?
Code:
x = 1
if x = 0
   err
end if    
It is clean in all aspects - the values fall into surely supported range and the numerical variables are clean from resolving point of view if you do not forward-reference them.
l_inc wrote:
OK. That's convincing, but I still don't understand, why the behaviour should differ between the provided overflow cases.
The behavior doesn't have to differ, this is just my heuristic. I made fasm put a little more effort into trying to recover from error when it is a substraction/addition overflow, because I consider it to be probable that such error is temporary in some non-synthetic case, and I don't think it is so probable with multiplication/shifting.

l_inc wrote:
As long as fasm didn't get a consistent value for x at the second pass it should continue with more passes. If it fails to continue, then the specific reason for that is that "a consistent value for x could not be found within n passes", if you prefer to give an exact error message.
Since fasm was not able to find the right solution, it is true that "a consistent value for ### could not be found" where for ### you can put just about anything that is defined in source. Which one of those was the crucial one may not be decidable in general. And even if some value appeared to be consistent, it may not have been consistent with what value it truly should have had for the correct solution.

The resolving of such source could be improved easily, so that fasm would be able to find the right solution in this and similar cases - but I try to avoid meddling with fasm's resolving algorithm unless it is absolutely necessary for some real appliance (revolution knows something about it - he had to make modifications to fasm's resolving engine for the purpose of ARM version, which has some encoding quirks that are way beyond the problems that x86 usually cause). Any change in those algorithms may allow fasm to find solutions in some case in which earlier it could not, but at the same time make it not find some solutions that it was finding easily before the changes.a consistent value for ### could not be found
Post 15 Mar 2012, 21:50
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Code:
the values fall into surely supported range    

IMHO your subdivision into surely and unsurely supported ranges is very uncomfortable. There are supported values, that any numeric variable may take at some time, and any other value is not supported. However as long as you changed the error source in the last release which hides the "false positive", achieving the err directive has no effect from the user point of view, so this detail about falling into "surely supported range" seems to be irrelevant.

Quote:
I made fasm put a little more effort into trying to recover from error when it is a substraction/addition overflow, because I consider it to be probable that such error is temporary in some non-synthetic case, and I don't think it is so probable with multiplication/shifting.

Could you provide an example (no matter synthetic or non-synthetic, because a correct implementation should not differentiate such things), where fasm is able to recover? Because as for me the heuristic literally kills fasm Smile :
Code:
if @F = 1
use16
else
use32
end if
xchg eax,eax
@@:

x = (-1) shl 64
x = -x    

Quote:
Which one of those was the crucial one may not be decidable in general. And even if some value appeared to be consistent, it may not have been consistent with what value it truly should have had for the correct solution.

If there are multiple valid solutions, it's also not generally decidable, which one should be chosen. However you still choose one. Therefore one possible option is to choose any variable, that did not achieve a consistent value at the last pass (i.e. it's value has changed during the last pass). It would be anyway dependent on that "crucial" one.

Another option is to relax the exactness of the message to smth. like "not all numeric values could be consistently resolved within n passes".
Post 15 Mar 2012, 22:30
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7724
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
(...) so this detail about falling into "surely supported range" seems to be irrelevant.
You asked whether it is clean - so the fact that it will not generate an overflow/syntax/etc. error is also relevant.

l_inc wrote:
Could you provide an example (no matter synthetic or non-synthetic, because a correct implementation should not differentiate such things), where fasm is able to recover.
Finding a correct solution in every case is not possible, because it is not a decidable problem. For that reason we are condemned to have to choose finding what solutions is more important to us - and of course the usual code always has the priority.
I do not have any real life example for when this heuristic becomes needed though; I made it as a "just in case" additional precaution.

l_inc wrote:
Because as for me the heuristic literally kills fasm Smile :
Code:
if @F = 1
use16
else
use32
end if
xchg eax,eax
@@:

x = (-1) shl 64
x = -x    
Oh, thanks for the bug report. It not caused by that heuristic, though. It is because of my today's changes to error message processing. I'm going to make a silent update. Wink

l_inc wrote:
If there are multiple valid solutions, it's also not generally decidable, which one should be chosen. However you still choose one. Therefore one possible option is to choose any variable, that did not achieve a consistent value at the last pass (i.e. it's value has changed during the last pass). It would be anyway dependent on that "crucial" one.
In any big program it then may be just any random label from the few thousands. What is the value of such message then? Would be as good as just "code cannot be generated".

l_inc wrote:
Another option is to relax the exactness of the message to smth. like "not all numeric values could be consistently resolved within n passes".
But that would mean, as I already noted previously, that all errors could end up with such message, and thus fasm would ultimately behave as it had just this one error message for all the situations.
Post 15 Mar 2012, 22:48
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar.

Powered by rwasa.