flat assembler
Message board for the users of flat assembler.

Index > Compiler Internals > Full path to the last included file. Suggestion.

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Good day. The idea is to add a preprocessor directive !p (similar to the %t symbol, just starts with a different non-symbol character ! to oppose to the assembly-stage expansion). The symbol !p should be expanded into a string, containing the full path to the last included source file (as usual unless it was redefined with the fix directive) within the current one. Thus if a.asm includes b.inc and b.inc includes c.inc, then after the line include 'b.inc' the symbol !p is expanded into '[path_to_b.inc]/b.inc', not into '[path_to_c.inc]/c.inc' even though b.inc included c.inc. If no files were included by the current source file, then !p should be expanded into the full path to the current source file. Whether the path is lowercased should possibly depend on the host OS.

This would allow to solve the "includeonce" problem, which is however not the only application of this feature.

Further improvement could be an introduction of a directive fakeinclude with one optional parameter which does not include anything, but sets the !p as though the normal include directive were invoked. If the fakeinclude parameter is omitted, then !p should be reset to the full path to the current source file.

Are there any contras?
Post 14 Apr 2012, 18:58
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7795
Location: Kraków, Poland
Tomasz Grysztar
If no other path elements than those specified in source were used to compose such value, it would probably not break the SSSO principle - so I don't see any obvious problems with such idea. I may try implementing something like this in 1.71, if I find a good way to weave it into preprocessor (and if I don't find any problems I don't see at the moment Wink).
Post 14 Apr 2012, 19:06
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Thank you for considering the suggestion.
Quote:
If no other path elements than those specified in source were used to compose such value, it would probably not break the SSSO principle

Do the environment variables (including the variable Include) belong to those path elements specified in the source?
Post 14 Apr 2012, 19:17
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7795
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
Do the environment variables (including the variable Include) belong to those path elements specified in the source?
They would have to stay unexpanded in that value.
Post 14 Apr 2012, 19:19
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
They would have to stay unexpanded in that value.

Well. That's not the idea behind the suggestion, because then introducing !p would bring nothing. The mentioned "includeonce" problem can be solved only when comparing expanded full paths. E.g. I'd like to be able to pass the value of !p to the file directive, cause include 'win32a.inc' works fine because of prepending one of the paths from the Include variable, but file 'win32a.inc' would not make use of the Include variable and therefore would not work.
Post 14 Apr 2012, 19:31
View user's profile Send private message Reply with quote
gunblade



Joined: 19 Feb 2004
Posts: 209
gunblade
This feature isnt really something i'd want - but I tend not to use include files much..

I was thinking though.. would it not be possible to implement something similar to what most C header files do, with the:

Code:
#ifndef __UNISTD_H
#define __UNISTD_H

;rest of header file here

#endif    


but using match instead?

(Just a suggestion - I havent tried this, so there may be multiple reasons why it may not work..)

EDIT: Oh.. sorry - just checked the link you posted, and it looks like you did do something similar.. although you seemed to have gone about it a long way..

Hmm - was trying to write a basic short macro that would do that - but i cant because I cant find a way to match a undefined constant, and without that.. you cant do the ifndef trick..

I've tried to find a way, but it doesnt seem to be possible to do a match on a constant which currently doesnt exist - i mean, it assembles.. but it always assembles false, theres no special word or symbol that can be used to do "if not defined" using match?
Post 15 Apr 2012, 00:32
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
gunblade
That's not a problem to prevent include duplication with the same approach as in C:
Code:
;first.inc
match =_first_inc,_first_inc { define _first_inc +

;header code

}    

But this way is not enough flexible. Some projects may require to include a file only once, but some projects may need to include the same file multiple times. The standard include files do not use the above approach, but I still may need to be able to prevent their duplication without modifying them.

Anyway, the topic is not about preventing include duplication. It's just one of the possible applications of the !p directive. Another one could be a self-compilation, where I need to make some project to be compiled into it's own source (possibly with slight modification), which requires to redefine include directive to partially perform as the file directive. This also can be combined with in-place-compilation, where the main source file is modified by the compilation process and in this way transformed into another source (I used that for example to obtain the value of %t during preprocessing stage of the second compilation).
Post 15 Apr 2012, 01:54
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7795
Location: Kraków, Poland
Tomasz Grysztar
gunblade wrote:
Hmm - was trying to write a basic short macro that would do that - but i cant because I cant find a way to match a undefined constant, and without that.. you cant do the ifndef trick..

I've tried to find a way, but it doesnt seem to be possible to do a match on a constant which currently doesnt exist - i mean, it assembles.. but it always assembles false, theres no special word or symbol that can be used to do "if not defined" using match?
There are a few ways to do it, probably the most simple one is this: http://board.flatassembler.net/topic.php?p=106862#106862

Alternatively, if you want to match the undefined constant, you can just match it with its name:
Code:
match =SOME_CONSTANT,SOME_CONSTANT {
  define SOME_CONSTANT something different
  ; this will not get assembled next time
}    
Post 15 Apr 2012, 07:31
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7795
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
E.g. I'd like to be able to pass the value of !p to the file directive, cause include 'win32a.inc' works fine because of prepending one of the paths from the Include variable, but file 'win32a.inc' would not make use of the Include variable and therefore would not work.
Wouldn't it be simpler to just make "file" use the same file searching rules as "include"?
There was such suggestion already and I was anyway thinking about following it.
Post 15 Apr 2012, 07:35
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
Wouldn't it be simpler to just make "file" use the same file searching rules as "include"?

I don't know, how much simpler this is, but I tried to propose a more general solution, that would solve a wider range of problems. Your last improvement is valuable, but for example the "includeonce" problem is still not solved by this modification. Actually also the idea of self-compilation did not give expected results, but that's my fault. I expected, that this should work for almost any project (in-place compilation also used):
Code:
format binary as 'asm'
include 'selfcompile.inc'
include 'main.asm'
    

where main.asm is the main source file of some project and selfcompile.inc looks as follows:
Code:
;Compiles all includes together into a source file
include fix include_directive_modified
include_directive_original fix include

included_files_list    equ
included_files_output equ ''
include_file_output_indentation equ ''
macro def_include_directive_modified
{
 macro include_directive_modified arg*
       \{
            tmp equ included_files_output
                       restore included_files_output
                       included_files_output equ tmp,include_file_output_indentation,"'",arg,"'",' included',13,10
         restore tmp
         
            match '',include_file_output_indentation \\{ if 0 \\}
               
            def_include_directive_modified
              include_file_output_indentation equ include_file_output_indentation,'   '
                 include_directive_original arg
              restore include_file_output_indentation
             purge include_directive_modified
            
            tmp equ included_files_list
                 restore included_files_list
                 included_files_list equ tmp arg
             restore tmp

             match '',include_file_output_indentation
          \\{
                  end if
                      display included_files_output
                       match list,included_files_list \\\{ irps f,list \\\\{ file f \\\\} \\\}
                   restore included_files_list,included_files_output
                   included_files_list equ
                     included_files_output equ ''
              \\}
  \}
}
def_include_directive_modified
db 'include fix include_directive_modified',13,10
db 'macro include_directive_modified arg* {}',13,10    

In fact the latest fasm release does compile this as expected, but the resulting source is compiled identically to the main.asm only if every file of the "include-tree" includes all it's descendants at the very beginning, which is not even true for the standard headers (e.g., win32a.inc defines TCHAR before including "equates"). So self-compilation is implementable, but the implementation is going to be much more complicated, than I expected.

Quote:
This is in fact a bug fix, because fasm's manual was stating that the same rules apply to both of them, but it was not true.

I actually never considered this to be a bug, because previous documentation versions never mentioned the "Include" variable usage for the file directive; and the manual statement: "These rules concern also paths given with the file directive" — was applied in the context of explicit environment variables specification (with surrounding % signs) and searching within the main source directory.

Regarding your concerns about the SSSO principle I never understood, why it can't be violated. Changing the "Include" variable also changes the output by using different include files without any modifications to the main source file. Even if you consider the include files as a part of that second "S" (which however reduces the value of the SSSO principle, because header files are often not distributed together with the main source components), then SSSO is still violated by the file directive, because replacing the included file changes the output without changing the source.
Thus IMHO any usage of external files already violates the SSSO principle, and !p would be just a part of the mechanism for external files handling.
Post 15 Apr 2012, 12:25
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7795
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
Quote:
This is in fact a bug fix, because fasm's manual was stating that the same rules apply to both of them, but it was not true.

I actually never considered this to be a bug, because previous documentation versions never mentioned the "Include" variable usage for the file directive; and the manual statement: "These rules concern also paths given with the file directive" — was applied in the context of explicit environment variables specification (with surrounding % signs) and searching within the main source directory.
I thought that it was what I wrote but when I read this section of documentation again I realized that it was not the obvious way to understand it. Quite the opposite - I feel that natural expectation was that the file searching works identically in both cases. I have left this statement there and now it really means that everything in treated the same in both cases.

l_inc wrote:
Regarding your concerns about the SSSO principle I never understood, why it can't be violated. Changing the "Include" variable also changes the output by using different include files without any modifications to the main source file.

This changes the source which you feed into assembler, but for the given contents of source assembler has to behave identically. The way assembler finds the source to assemble is system-dependent, because the file names and paths are system-dependent. Running "fasm my.asm" will get different source depending on what directory you run it in. So the SSSO rule applies to the contents of the source and not to the system specific of where this source comes from. Including files is just composing many sources into one - what files will get used to compose the source may be system-dependent, but how the final source is then assembled, is not.

l_inc wrote:
(which however reduces the value of the SSSO principle, because header files are often not distributed together with the main source components)
Yes, it reduces the value of SSSO a bit, but the dependence on filesystem that is OS-dependent isn't something that we can get rid of if we want to allow multi-file projects. Well, there is other solution, the one than Betov used in his "SpASM" back in the early 2000's, but it was way too unusual. And if you distribute a source of project that was written with non-standard headers, you should include all the headers in the project - that's the rules for program in "Examples" section, for instance.

l_inc wrote:
(...) then SSSO is still violated by the file directive, because replacing the included file changes the output without changing the source.
The input to "file" directive can also be considered the source - the binary source in this case. Again, what source contents are found on the given system is something that is very system-dependent, but for the given set of source files, as long as you are able to ensure that the same files are provided under the given names by the system, the SSSO rule still applies.

And, speaking of "includeonce" problem - consider, for example, that you have a hard link in filesystem, so you have one file under two different names. That's another system-dependent thing, and there isn't much you could do about it.
Post 15 Apr 2012, 13:17
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
And, speaking of "includeonce" problem - consider, for example, that you have a hard link in filesystem, so you have one file under two different names. That's another system-dependent thing, and there isn't much you could do about it.

I thought about it (no matter if that's a soft link or a hard link), and yes there's not much I could do about it, but it's also less commonly used. Anyway I don't think !p is absolutely necessary. That was just an idea.
Quote:
as long as you are able to ensure that the same files are provided under the given names by the system, the SSSO rule still applies

Well... then adding !p also does not break SSSO. You'd just need to extend the SSSO definition with "as long as you are able to ensure the same environment variable values are provided under the given names by the system" which is apropos already partially included by the need to provide same files (consider for example file '%WinDir%\system32\kernel32.dll'). IMHO SSSO borders are already crossed, you just decide how far.
Post 15 Apr 2012, 15:06
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7795
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
You'd just need to extend the SSSO definition with "as long as you are able to ensure the same environment variable values are provided under the given names by the system" which is apropos already partially included by the need to provide same files (consider for example file '%WinDir%\system32\kernel32.dll'). IMHO SSSO borders are already crossed, you just decide how far.
That would make the environment variables part of the source. Currently they are not source themselves, they are just used to help to find the right source in your OS (and they hide some the OS-dependent layer of finding the right files). Just like the path you provide as an argument for command-line fasm is not a source code itself, but a system-dependent way of telling fasm where the source is.
Post 15 Apr 2012, 15:15
View user's profile Send private message Visit poster's website Reply with quote
gunblade



Joined: 19 Feb 2004
Posts: 209
gunblade
Tomasz Grysztar wrote:
Alternatively, if you want to match the undefined constant, you can just match it with its name:
Code:
match =SOME_CONSTANT,SOME_CONSTANT {
  define SOME_CONSTANT something different
  ; this will not get assembled next time
}    


I did not know you could do that, thanks... Very Happy It's also not mentioned in the documentation? might be worth a mention, can be a useful feature to see if a constant is defined (for default values that can be overridden/etc).

Thanks l_inc too for posting that reply - althought it didnt make sense to me until tomasz mentioned the "trick" of matching a variable with itself..
Post 15 Apr 2012, 23:00
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7795
Location: Kraków, Poland
Tomasz Grysztar
gunblade wrote:
I did not know you could do that, thanks... Very Happy It's also not mentioned in the documentation? might be worth a mention, can be a useful feature to see if a constant is defined (for default values that can be overridden/etc).
All the rules for the "match" directive are documented in manual - there is only a few of them. More examples of its applications (including the text I once wrote on this board) I planned to include in the final form of Understanding fasm article (perhaps after 1.70 is released I can get back to it). But there is really nothing special about any of them - the same simple rules apply to each one.

PS But the Borsuc's method I linked to is simpler and thus better.
Post 16 Apr 2012, 07:46
View user's profile Send private message Visit poster's website Reply with quote
gunblade



Joined: 19 Feb 2004
Posts: 209
gunblade
Sorry to drag this on.. but I seem to be misunderstanding something here.

I understand now that an undefined constant will resolve to its own name, so I see how your method would work by checking the constant with its own name (with the =, so that it is taken as a string). However.. Borsuc's method is strange.. it works, but I dont see how/why it does.

His method uses:
Code:
match x,FILE_INC {    


Where he says that:
Quote:
and the 'x' is used to "match anything" -- in other words, it matches any symbol(s) available, which happens all the time unless the symbolic constant is empty...


However.. two things wrong with that (as far as i can see):
1) Theres no mention of x being a special variable that is interpreted as "match any".
2) This also seems to work with ANY undefined variable on the left.. so for example, all of these seem to match (but they shouldnt?):

Code:
match y,FILE_INC {
match aaaa,FILE_INC {
match asdf, FILE_INC {    


Should both of those names not resolve to text-equivalent of their own constant names.. so you'd essentially be doing: if ("y" == "FILE_INC") (.. i know that doesnt work and that you'd have to use strcmp, but its not proper C Smile) which should return false, so the match should not happen?

Am I missing some small detail - are the "constants" on either side of the comma not expanded in the same way?

(Using fasm 1.69.52 - just in case its a bug in the preprocessor (probably not, i'm probably just misunderstanding this entirely Very Happy)).

EDIT: Hm... looks like you may be using this feature of the match, if i understand right:

Quote:
Now what happens, when the name symbol is not preceded with = in the expression to be matched? Back to the manual:
fasm manual, section 2.3.6 wrote:
If some name symbol is placed in the pattern, it matches any sequence consisting of at least one symbol and then this name is replaced with the matched sequence everywhere inside the following block, analogously to the parameters of macroinstruction. For instance:
Code:
    match a-b, 0-7
     { dw a,b-a }    
will generate the dw 0,7-0 instruction.


So for the duration of the code inside the match{} (the first time round, when FILE_INC is undefined, and hence equal to "FILE_INC"), then x (or y, or aaaa), will be defined as "FILE_INC"? whereas once FILE_INC is defined, and blank, then x (or y...) will not have any sequence/string to match against, and so the match will fail?
Post 16 Apr 2012, 13:52
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7795
Location: Kraków, Poland
Tomasz Grysztar
gunblade wrote:
So for the duration of the code inside the match{} (the first time round, when FILE_INC is undefined, and hence equal to "FILE_INC"), then x (or y, or aaaa), will be defined as "FILE_INC"? whereas once FILE_INC is defined, and blank, then x (or y...) will not have any sequence/string to match against, and so the match will fail?
Yes. Please read that detailed explanation of MATCH which I already linked to. It explains it more specifically than the manual (see the last paragraphs of my first post there).
Post 16 Apr 2012, 14:13
View user's profile Send private message Visit poster's website Reply with quote
gunblade



Joined: 19 Feb 2004
Posts: 209
gunblade
Oh.. whoops, I stopped reading after the bit that I quoted.. You explain it perfectly in the last paragraphs..

It seems like a strange way of doing things in my opinion.. the fact that match can do both "checking" and "defining", depending on how you write it.. Not a bad idea, just seems like a strange combination.

I understand it well now though, thanks for the the replies - and sorry l_inc for barging in on your topic Smile But to get back on topic.. In my opinion theres no need for the !p since match can do the job nicely. If you do want to include a file multiple times, you could simply purge/undefine the constant before you include it the second time.. so:
Code:
include 'file.inc'
purge FILE_INC
include 'file.inc'    

.. granted, its not the neatest solution, and requires you to check what the constant that the include file will define is.. but if you stick to "standard" naming (filename, in caps, replace any . with _) then its easy enough to do..

But as i say.. I dont use include files as much as you probably do.. so its not a big issue for me - but could be more important for you (and others) who use that feature more.

And thanks again Tomasz.. took me a while but I got there Smile
Post 16 Apr 2012, 14:24
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
gunblade
Quote:
In my opinion theres no need for the !p since match can do the job nicely. If you do want to include a file multiple times, you could simply purge/undefine the constant before you include it the second time.. so:

First of all, to keep your include files more or less clean with this approach, for every header you'd need to create an interface-header file, which looks as follows:
Code:
match x, _FILE_INC
{ 
  define _FILE_INC
  include 'file.inc'
}    

Otherwise, you'll need to escape many symbols with a backslash. It's probably not a problem for the (closing) braces, but it's very easy to forget to additionally escape # or `.

Again, what do you do with standard headers and a plenty of 3rd party headers, that do not follow this approach? Modify them all? Kind of a bad idea.

P.S. I absolutely accept the Tomasz Grysztar's explanation, why !p is not a perfect idea.
Post 16 Apr 2012, 19:48
View user's profile Send private message Reply with quote
gunblade



Joined: 19 Feb 2004
Posts: 209
gunblade
Hmm - good point, didnt consider the backslashing required since it would be inside a bracket..

what about a special command that would stop the preprocessor processing the current file?

Then you could do:

Code:
match , _FILE_INC {
stop_preprocessor
}
define _FILE_INC

.. rest of header code..    


doesnt solve the 3rd party header problem - but there arent that many at the moment (bar the one in fasm itself, and a couple of libraries released by people on this board), so wouldnt be a massive issue to add them (even if you do it yourself on your own copy of the 3rd party code..)

Dunno - just a suggestion.. stopping pre-processing might be a hard/bad idea - its just one possible (if ugly) solution.
Post 17 Apr 2012, 00:38
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on GitHub, YouTube, Twitter.

Website powered by rwasa.