flat assembler
Message board for the users of flat assembler.

Index > Compiler Internals > Suggestion: Save Virtual Block as File

Goto page 1, 2, 3  Next
Author
Thread Post new topic Reply to topic
m3ntal



Joined: 08 Dec 2013
Posts: 296
m3ntal 18 Jun 2014, 17:14
FASM can load external source+binary files (include+file), but there is no way to save files as .EXT at assembly time. You must assemble the entire project as .EXT (format binary as '.EXT'). Example:
Code:
savefile p, 'NAME.EXT'    
This is merely a suggestion. And I only care what Tomasz thinks on this issue.


Last edited by m3ntal on 19 Jun 2014, 20:01; edited 2 times in total
Post 18 Jun 2014, 17:14
View user's profile Send private message Reply with quote
baldr



Joined: 19 Mar 2008
Posts: 1651
baldr 18 Jun 2014, 18:05
And there is m3ntal, who probably knows about e-mail and private messages more than anybody else but don't use them to contact the particular person. Wink
Post 18 Jun 2014, 18:05
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland
Tomasz Grysztar 18 Jun 2014, 20:31
m3ntal wrote:
FASM can load external source+binary files (include+file), but there is no way to save files as .EXT at assembly time. You must assemble the entire project as .EXT (format binary as '.EXT'). Example:
Code:
savefile p, 'NAME.EXT'    
This is merely a suggestion. And I only care what Tomasz thinks on this issue but I will not respond.
Is "p" in your example the addressing space label (one defined with "::")? That would make sense, and such directive would then dump contents of entire addressing space into file. I think this is a nice design, and should be easy to implement within current fasm's implementation*.

But I'm not sure about the syntax. Maybe something like:
Code:
store p as 'name.ext'    
This would introduce no new reserved words, and I think I like it.
________________
* Though it would sometimes hit some performance problems when having multiple passes but saving file early in the source, when assembler does not yet know that it will need another pass.
Post 18 Jun 2014, 20:31
View user's profile Send private message Visit poster's website Reply with quote
m3ntal



Joined: 08 Dec 2013
Posts: 296
m3ntal 18 Jun 2014, 21:36
Quote:
Is "p" in your example the addressing space label (one defined with "::")? That would make sense
Yes, p:: then p:
Quote:
But I'm not sure about the syntax. Maybe something like:
Code:
store p as 'name.ext'    
Good idea. Much better to use/improve existing keywords than to reserve names.

Thanks for your response.
Post 18 Jun 2014, 21:36
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 18 Jun 2014, 23:04
Tomasz Grysztar
I think, spawning arbitrary files at arbitrary file system locations is a bit too far from what a compiler should do. I would definitely agree, that this possibility could be useful for many use cases, but those are not really compiler related. Like the next ideas could be to create folders, to write into the registry or to call arbitrary external library functions, which are no doubt useful but those are rather tasks for some scripting languages like shell or autohotkey.

I'm not sure I want to be afraid of compiling some unknown fasm sources, because the very compilation could lead to polluting my system with hundreds of fat files.

_________________
Faith is a superposition of knowledge and fallacy
Post 18 Jun 2014, 23:04
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20462
Location: In your JS exploiting you and your system
revolution 19 Jun 2014, 00:59
m3ntal wrote:
(PS: There are users who are not experienced enough in programming to understand why this feature would be useful and there are knowledgeable programmers who are good at answering questions but they don't have the creativity to imagine the possibilities. And there are users who have knowledge of macros but can't seem to understand the difference between an immediate and displacement. That's why Z6502 needs the =i prefix).
So for dummies like myself please explain the usefulness. Since the rest of us are not in your exclusive club of extreme mindreading we feel left out.

BTW: How is this related to macros and immediate displacements? Seems to be a non sequitur.
Post 19 Jun 2014, 00:59
View user's profile Send private message Visit poster's website Reply with quote
m3ntal



Joined: 08 Dec 2013
Posts: 296
m3ntal 19 Jun 2014, 03:04
I think the only additional feature that's needed is the ability to check if the file currently exists (ifexists?).

revolution: Not referring to you. I'm the real dummy who has a horrible addiction to programming when I should be concerned about making money and taking care of responsibilities. To answer your question, though, there are many cases where I've needed to save a block of memory as "FILE.EXT" - utilities, conversions, create resources, make files, etc.
Post 19 Jun 2014, 03:04
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20462
Location: In your JS exploiting you and your system
revolution 19 Jun 2014, 03:39
m3ntal wrote:
To answer your question, though, there are many cases where I've needed to save a block of memory as "FILE.EXT" - utilities, conversions, create resources, make files, etc.
Sure. I'm always finding this also, but I can't quite grasp how this fits with an assembler?

It would seem to be similar to edfed's suggestion where we could create arbitrary files at compile time.
http://board.flatassembler.net/topic.php?t=15850
Post 19 Jun 2014, 03:39
View user's profile Send private message Visit poster's website Reply with quote
m3ntal



Joined: 08 Dec 2013
Posts: 296
m3ntal 19 Jun 2014, 05:15
Same argument can be used against the "if" directive or "file" (which could insert malicious code into the executable).

"An assembler shouldn't do that" - FASM is *not* just another typical assembler. It is unique and can do things that other assemblers can't. For example, FASM macros have multiple inheritance that goes beyond C++/Java's concept of OOP.
Post 19 Jun 2014, 05:15
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20462
Location: In your JS exploiting you and your system
revolution 19 Jun 2014, 05:24
m3ntal wrote:
"An assembler shouldn't do that" - FASM is *not* just another typical assembler. It is unique and can do things that other assemblers can't. For example, FASM macros have multiple inheritance that goes beyond C++/Java's concept of OOP.
But that is not an argument. You could also say that we should have a browser and a twitter app included in fasmw because fasm can do things other assembler cannot. If you could present a usage case that shows the utility of it then I would be grateful to understand where this is used.
Post 19 Jun 2014, 05:24
View user's profile Send private message Visit poster's website Reply with quote
evk1



Joined: 18 Jun 2014
Posts: 24
evk1 19 Jun 2014, 05:33
It can be a good idea. But what about saving files to another folders?

Code:
store p as 'folder\name.ext'    


Then we need directory creating and directory existence check directives too.

_________________
Sorry for my English
Post 19 Jun 2014, 05:33
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 19 Jun 2014, 06:13
This idea raises some security issues. For example, one can hide malicious code in his source code and then save it to somewhere on the system, masking the infiltration with some useful program compiled from the source. Even worse, if the source is obfuscated good enough and looking like a library, it can be reused by beginners and this way spreading the malicious code even further.
Post 19 Jun 2014, 06:13
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
m3ntal



Joined: 08 Dec 2013
Posts: 296
m3ntal 19 Jun 2014, 06:33
revolution: You misunderstood. I'm saying that it's false categorization to throw FASM in the same boat with other assemblers because FASM is different. My reason for making this suggestion is: FASM has the ability to load external files but it can't save.

It's not an argument to say, in other words, "An assembler should not do that". Everyone has their own ideas of what an assembler is supposed to do based on their definition of what assembler is and their limited experience. Who cares? I'm only concerned about good ideas.

evk: You answered your question Smile 'folder\file.ext', '..\..\file.ext'. (Awesome examples in non-x86!).
Post 19 Jun 2014, 06:33
View user's profile Send private message Reply with quote
m3ntal



Joined: 08 Dec 2013
Posts: 296
m3ntal 19 Jun 2014, 06:49
John: Programmers can hide malicious code in runtime examples, too. If you don't trust someone, just don't download or run their code. It's that simple.
Post 19 Jun 2014, 06:49
View user's profile Send private message Reply with quote
evk1



Joined: 18 Jun 2014
Posts: 24
evk1 19 Jun 2014, 06:51
m3ntal: I just want to say if we add such feature in FASM then we also have to add directives for creating directories, checking directory existence, checking file existence and maybe even for listing and removing directories. The idea is good, but as you can see there are many questions about implementation.

_________________
Sorry for my English


Last edited by evk1 on 19 Jun 2014, 07:02; edited 1 time in total
Post 19 Jun 2014, 06:51
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland
Tomasz Grysztar 19 Jun 2014, 06:59
A similar feature was requested on several occasions, for a various reasons. At the time when there was variant of LOAD directive that could load directly from files, it was suggested that there should be analogous STORE feature. For the most of the time it did not appeal to me, but I think it could be nice to be able to track the contents of some virtual blocks.

The security concerns are valid (it reminds me of the time when someone did make a web-based fasm feature and I pointed out to him that I could assemble something like "file '/etc/passwd'"), but I think this could be remedied by requiring this feature to be explicitly enabled with a command line switch.

As for the checking for file existence, creating directories, etc. - no, I don't think any of this would be good idea, that's what the batch/script/make files are for. What appeals to me in this feature is not the ability to mess with file system, but the ability to store some additional information beside the main output format. The DISPLAY is there for this purpose, but ability to create supplementary files could sometimes make one's life easier.
Post 19 Jun 2014, 06:59
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland
Tomasz Grysztar 19 Jun 2014, 07:03
evk1 wrote:
It can be a good idea. But what about saving files to another folders?

Code:
store p as 'folder\name.ext'    


Then we need directory creating and directory existence check directives too.
I strongly disagree. When you specify output path for assembler you can also make it in another folder. When you use INCLUDE or LOAD it can also read something from a different folder. It is natural that you need to have the right environment set up before assembling.
Post 19 Jun 2014, 07:03
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 19 Jun 2014, 10:21
Tomasz Grysztar
Quote:
but I think this could be remedied by requiring this feature to be explicitly enabled with a command line switch

This goes the direction, when one needs to try some command line arguments in order to find out, how a source should be compiled. I.e. the reason behind the emergence of the SSSO principle would become invalidated.

As an alternative I'd rather suggest to define a specific folder using an environment variable (similar to Include), the value of which is allowed to be the only destination of such stores. If the variable is not defined, then current folder is used. No path (not even relative path) arguments should be allowed to be passed to such new stores.

Quote:
ability to create supplementary files could sometimes make one's life easier

I definitely remember requests that could be gratified by this. Like automatic build number incrementation. Or randomization of the generated code between different compilations: %t is a way to go, but this topic's suggestion would allow to implement a real PRNG. Still not sure, if this is a good idea.

Anyway if you're going to implement this, there are a number of things to consider. The current way of the output generation is a multipass convergence against a solution satisfying a number of declarative constraints imposed by the source code. The new store directive is rather an imperative construct making the compiler create a file, which makes it quite different to what the normal compilation output is.

Considering the above I see the implementation as creating an in-memory copy of an addressing space in question each time the new store directive is encountered. A pass failure or the whole compilation failure discards all the copies created this way, so that no files are created. The files must be created in-order, so that an order of multiple stores to the same file is defined. Another question is whether the load or even include accesses should be redirected to the in-memory copies as well as how to tell these directives to read from the new store destination directory.

_________________
Faith is a superposition of knowledge and fallacy
Post 19 Jun 2014, 10:21
View user's profile Send private message Reply with quote
m3ntal



Joined: 08 Dec 2013
Posts: 296
m3ntal 19 Jun 2014, 15:05
I respect Tomasz's opinions.

evk: I knew they wouldn't go for that. If there was a preprocessor language that supports file I/O, it'd have to use a unique prefix such as ? on all directives to prevent conflict with runtime names: ?open, ?create, ?write, ?exists, etc. Multi-pass would make it hard to implement these features. Remember, we can make our own assemblers, too, and edited versions of FASM.

l_inc:
Quote:
No path (not even relative path) arguments should be allowed to be passed to such new stores.
Why? What's wrong with specifying a folder like \BINARY\ or ..\? "file" doesn't have this limitation. For security? How do you ensure the same security at runtime? (CreateFile, fopen).
Quote:
those are rather tasks for some scripting languages...
Umm... if I'm not mistaken, FASM language is an interpreter.
Post 19 Jun 2014, 15:05
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland
Tomasz Grysztar 19 Jun 2014, 15:36
l_inc wrote:
This goes the direction, when one needs to try some command line arguments in order to find out, how a source should be compiled. I.e. the reason behind the emergence of the SSSO principle would become invalidated.
The behavior of this directive would not affect the output of assembly, hence the SSSO principle is irrelevant here. It's like when you choose (from command line) to redirect the DISPLAY output into a file, or nul - it does not have any effect on the main assembly output. With this kind of STORE the effect would be similar, unless the assembler wrote the contents of file early enough that it could affect some LOAD - but I think such behavior should be undefined and this kind of usage discouraged - the moment when final write occurs may vary depending on implementation (e.g. for performance reasons). Probably the best option would to buffer all the data and write it at the same time when the DISPLAY buffer is flushed - this way I would avoid the problem of writing the same file many times during the consecutive passes.

As for the command line option, I thought that it could just make this STORE variant do nothing, like when you redirect DISPLAY output to nul.
Post 19 Jun 2014, 15:36
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2, 3  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2025, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.