flat assembler
Message board for the users of flat assembler.

Index > Compiler Internals > Full path to the last included file. Suggestion.

Goto page Previous  1, 2
Author
Thread Post new topic Reply to topic
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
gunblade
Quote:
what about a special command that would stop the preprocessor processing the current file?

I assume, stopping the preprocessor should mean eating up the remaining part of the file. Otherwise problems with the following assembly stage will arise. And assuming that, I see at least one issue with this suggestion. You can't just stop preprocessing, because you have to decide what to do with the current expansion context. Even with your simple example there's a need in special handling of the closing brace. And closing braces are also often a result of macro expansion (unescaping is probably the most common example).
Quote:
but there arent that many at the moment (bar the one in fasm itself, and a couple of libraries released by people on this board)

Do you prefer to force yourself to redistribute even the standard headers just because of the very little modification of every file?
Post 17 Apr 2012, 12:55
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Btw. just noticed, that fasm is unable to properly handle incomplete macros at the end of an included file. For example, the following code
Code:
include 'incomplete.inc'
x    

makes fasm crash, if "incomplete.inc" contains just this line:
Code:
macro x {    
Post 17 Apr 2012, 16:03
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7796
Location: Kraków, Poland
Tomasz Grysztar
Thanks for finding this. One more bug fixed for 1.70 release. BTW: I was thinking about releasing it today, though I'm still hesitating whether to give it some more time just in case some new bug reports come in. All of the projects I tested seem to assemble correctly though.
Post 17 Apr 2012, 16:29
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7796
Location: Kraków, Poland
Tomasz Grysztar
When testing a fix for it, I created this funny little test. It shows that having cross-file macro boundaries can even become slightly useful if you really want it to. Wink

TEST.ASM:
Code:
include 'test.inc'

  hello db 'Hello!$'

begin

   mov  ah,9
   mov  dx,hello
   int  21h

end    


TEST.INC
Code:
macro begin {
  org 100h
}

begin fix } begin

macro end {
  int 20h    
Post 17 Apr 2012, 17:27
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
I'm still hesitating whether to give it some more time just in case some new bug reports come in.

Would it be acceptable for you to create a topic, where you post a release candidate, so that the community can test it within 24 hours (the deadline of course does not have to be strict) before you make it available through the download section? That would be however only helpful in case you incorporated new bugs. The old bugs won't probably be discovered this way.
Quote:
It shows that having cross-file macro boundaries can even become slightly useful

The profit (if any) seems to be very limited. Are you going to allow cross-file macros? This would get in conflict with current documentation:
Quote:
The whole included file is preprocessed before preprocessing the lines next to the line containing the include directive.
Post 17 Apr 2012, 19:50
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7796
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
Would it be acceptable for you to create a topic, where you post a release candidate, so that the community can test it within 24 hours (the deadline of course does not have to be strict) before you make it available through the download section?
Well, the 1.69.51 was posted as a release candidate. So we are in fact past the 24 hours. I'm already preparing the release.

l_inc wrote:
Are you going to allow cross-file macros? This would get in conflict with current documentation:
Quote:
The whole included file is preprocessed before preprocessing the lines next to the line containing the include directive.
There is no conflict there. The macro definition is created out of preprocessed lines, it doesn't matter how they were generated.
Post 17 Apr 2012, 19:56
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
The macro definition is created out of preprocessed lines, it doesn't matter how they were generated.

As far as I understand, the preprocessing is not completed until the macro definition is closed. Otherwise an incomplete macro definition should be a valid construction. Anyway as for me allowing cross-file macro definitions could hide mistakes and gives (almost) no profit instead.
Post 17 Apr 2012, 20:01
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7796
Location: Kraków, Poland
Tomasz Grysztar
Definition of macro consist of lines that already are preprocessed. Of course, some of the preprocessing is not applied to such lines, just as it is not applied to lines than contain other preprocessor's directives (like INCLUDE) - you can look at it as if each line of the macro definition was a line containing preprocessor's directive. After the definition is complete, it is used in preprocessing the later lines. As for why did you think that incomplete macro should then be a valid construction, I don't understand.
Post 17 Apr 2012, 20:08
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
Definition of macro consist of lines that already are preprocessed

Not completely. The preprocessing continues by taking into account the produced macro definition. According to (my understanding of) the documentation the included file is completely preprocessed before proceeding, which means, that all macro definitions were already recognized. If you allow an incomplete macro definition within the included file, that would mean, that preprocessing can be successfully completed with an incomplete macro definition, which makes it to a valid construction.
Post 17 Apr 2012, 20:21
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7796
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
Tomasz Grysztar
Quote:
Definition of macro consist of lines that already are preprocessed

Not completely. The preprocessing continues by taking into account the produced macro definition. According to (my understanding of) the documentation the included file is completely preprocessed before proceeding, which means, that all macro definitions were already recognized. If you allow an incomplete macro definition within the included file, that would mean, that preprocessing can be successfully completed with an incomplete macro definition, which makes it to a valid construction.
Preprocessing is the process of transforming the source lines according to given rules. The macro definition is something that is composed of the already transformed lines.

The same thing as with INCLUDE directive happens when you use a macroinstruction. As manual states for line generated by macro:
2.3.7 wrote:
(...) the newly generated line goes through the standard preprocessing, as described above.
So for invocation of macroinstruction the same thing is true as for including the file: all its lines get preprocessed before going further. The meaning of "preprocessing" is the same in both cases, it means generating the transformed text. And in the same way you can start definition of macro in lines generated by other macro, but continue its definition afterwards.
Post 17 Apr 2012, 20:35
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc

Quote:
Preprocessing is the process of transforming the source lines according to given rules.

Doesn't defining these rules belong to the preprocessing?
Quote:
The same this as with INCLUDE directive happens when you use a macroinstruction.

You try to explain it from the practical point of view, i.e. how it really is. I try to explain from the interpretation of the documentation:
Quote:
The whole included file is preprocessed before preprocessing the lines next...

How can you state, that the whole file is preprocessed, if the preprocessing can not be finished at that point? Considering the following example:
Code:
macro def_x { macro x { } def_x    

the macro def_x is completely preprocessed just as the line macro x { is generated. But the preprocessing of the file, which contains only this code cannot be completed because of the incomplete macro definition. That's how I once (being guided by exactly the above citation) explained to myself the inability of fasm to process cross-file macro definitions.
Post 17 Apr 2012, 21:12
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7796
Location: Kraków, Poland
Tomasz Grysztar
Well, I may have failed to state myself clearly in the documentation, but it was not my intention to state that preprocessing of the inside of file or a macro is a kind of separate preprocessing process. If that was the case, the you wouldn't even be able to use macroinstructions generated in one file in the other one.

The preprocessing, as it is stated in many places in various documents I produced, is performed one line at a time (that's in fact a very important concept for fasm's architecture - that it is without an exception a line-based processor, that's for example reason why there will never be an inline macro support in fasm). So you can look at a line in context of macro definition as a preprocessor's directive saying "take this line and add it to the definition of macro X, while generating no text for the assembler at all". So preprocessor keeps generating and preprocessing lines according to all its rules, one line at a time. The definition of macroinstruction consists of lines that have already been preprocessed and are now part of the definition - stored somewhere, doesn't really matter where, that's the detail of implementation - and not the part of the text generated for the assembler. But preprocessor can keep going, and generating and preprocessing the next lines, and until it finds a directive that tells it than the macro definition can be considered finished (the closing brace is such "directive") it will continue to preprocess lines in such a way, that they become part of the definition of macro. And if it hits the end of source and the definition has not been finished, it will let you know with an error message that the definition of macro has not been completed, because that is a nice thing to do.

And once again, the same what is said about the included file can be said about the macro:
Quote:
The whole content generated by macroinstruction is preprocessed before preprocessing the lines next to it.
It is phrased differently in documentation, but the meaning is the same.
Post 17 Apr 2012, 21:29
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
If that was the case, the you wouldn't even be able to use macroinstructions generated in one file in the other one.

From my point of view preprocessing of a file has two consequences:
1) Defined constants/macros/structures
2) Expanded constants and macroblocks.
According to the first consequence after a file preprocessing the constants and macros remain defined. Thus being able to use macroinstructions defined by another file perfectly fit into my idea of preprocessing.

Quote:
... And if it hits the end of source and the definition has not been finished, it will let you know with an error message that the definition of macro has not been completed, because that is a nice thing to do.

To conclude your explanation: a file is called to be preprocessed if and only if every it's line is individually preprocessed. And line preprocessing itself is described in the documentation. This simple definition does not require macroblocks to be closed within a file.

Within the whole explanation there is one word, which can be changed without any conflict (as it seems to me) with the rest of the explanation: "And if it hits the end of file and the definition has not been finished..." So why do you think it's better to message an error only when reaching the end of a source? To include files with incomplete macro definitions can be a dangerous thing:
Code:
include 'incomplete.inc'
include 'hello.inc'
Hello ;"error: illegal instruction". How the hell can it be illegal, if I defined this macro in my 'hello.inc'?!    

where 'incomplete.inc':
Code:
macro some_macro {
<much code with internal macro definitions and unclear stuff probably coded by aliens from the Andromeda Galaxy>
;} ;Oops. Just accidentally forgot to add (or unescape) a brace here.    

and 'hello.inc':
Code:
macro Hello { display 'Hello',13,10 }    

It would be much easier to find the mistake if the error message said the macro some_macro from 'incomplete.inc' is incomplete.
Post 17 Apr 2012, 22:23
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7796
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
It would be much easier to find the mistake if the error message said the macro some_macro from 'incomplete.inc' is incomplete.

Yes, I thought for a moment, that signalizing an error instead might sometimes be more practical - but I chose to follow the consistency of language, I simply saw no other way, it had to behave the same as in case of macro, because that's how I defined it. You may notice, that in "Understanding fasm" article I wrote:
Understanding fasm, Macroinstructions wrote:
When you invoke macro, however, and preprocessor uses the recipe to produce new lines, it also preprocesses all those new lines before it goes further - just like it is with INCLUDE directive.
It again gives away that I intended these cases to behave analogously to each other (with the important difference, that they use different "line makers", the concept also explained in that article).
Post 17 Apr 2012, 22:30
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
with the important difference, that they use different "line makers", the concept also explained in that article

I definitely need to continuously reread this article, continuously gaining more and more understanding. Smile

I actually wanted to mention an inconsistency (as it seems to me) with the processing of the backslash before you make 1.7 available, but forgot to do it timely. Backslash is not defined as a symbol character, so one may expect it can be a part of a user-defined symbol (if it's not unescaped by a macro expansion nor located at the end of a line), which is also confirmed by the following example:
Code:
\a equ 'Hello',13,10
display \a

\\\b = 5
if \\\b = 5
        display '5',13,10
end if    

On the other side, symbols like \, \\\\\\\ as well as \\\a\b or a\b are not handled properly. Furthermore, the backslash is even able to make symbol characters become user-defined symbols:
Code:
\/ equ 'Hello',13,10
display \/

\= = 65
display \=    
Post 17 Apr 2012, 23:08
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7796
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
I definitely need to continuously reread this article, continuously gaining more and more understanding. Smile
You may want to wait till I finish it and correct all possible mistakes (I still haven't ever re-read it thoroughly after writing). Also, the discussions like this in general influence how I decide to explain some things, I make take some misunderstanding into consideration.

l_inc wrote:
On the other side, symbols like \, \\\\\\\ as well as \\\a\b or a\b are not handled properly. Furthermore, the backslash is even able to make symbol characters become user-defined symbols.
I don't remember how well I documented it, but backslash character is a very special case, and very specific rules apply to its processing. So it is a "doubly special" character that requires a separate description.

PS I'm now looking at the "Understanding fasm" text and I see that backslash is explained completely there. Though I might add a sentence or two to those paragraphs, to clear things up unequivocally.
Post 17 Apr 2012, 23:19
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
I'm now looking at the "Understanding fasm" text and I see that backslash is explained completely there.

Sorry. You're right. What is IMHO not really clearly explained, is that it starts a new token. Thus it's a left(-associative) semisymbol character. Smile

P.S. Probably the main documentation file should incorporate your explanation, because the information belongs to basic definitions as opposed to just contributing to understanding. Currently the main documentation file states:
Quote:
Any of the+-*/=<>()[]{}:,|&~#‘is the symbol character. The sequence of other characters, separated from other items with either blank spaces or symbol characters, is a symbol.

And this does not seem to be correct.
Post 17 Apr 2012, 23:38
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on GitHub, YouTube, Twitter.

Website powered by rwasa.