use fasm as a generic tool

Index > Projects and Ideas > use fasm as a generic tool

Goto page 1, 2 Next

Author

Thread

edfed

Joined: 20 Feb 2006
Posts: 4333
Location: Now

edfed 15 Apr 2019, 07:31

something miss to fasm, the hability to execute commands from the source.

like

Code:

_shell "command arg, arg, arg"

of course, this functionality is not very optimal, but it let the programmer control a set of processes from only one source code.

for example, you can setup the generation of test files to be used with your program.
you can also generate the libs or secondary programs.

i currently work on c++ projects and it appears clearly the need to generate multiple files from a single source to let the programmer (me) think more about the funcitonnalities of the project than to think about the way to manage the sources.

the goal here is to find the best keyword to use to make this...

should it be F_SHELL? F_COMMAND? FASM_SHELL? OS_COMMAND? ext_script? shell_os_execute?

i don't know.
the keyword should be simple, and cannot collide with keywords still used in projects.

one the keyword choose, i'll try to implement it in my version of fasm.

_________________
Smile

fool gitlab fasmstuff
gitlab foolstuff

15 Apr 2019, 07:31

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8353
Location: Kraków, Poland

Tomasz Grysztar 15 Apr 2019, 13:22

There is a reason why "virtual as" feature only allows to specify a file extension from a limited charset. With a tool like assembler nobody expects that compiling a source could spawn files in arbitrary locations or - even worse - execute any shell commands.

However, you can use fasm/fasmg to generate a command line script, for example, and then execute it in a batch build process.

15 Apr 2019, 13:22

ProMiNick

Joined: 24 Mar 2012
Posts: 799
Location: Russian Federation, Sochi

ProMiNick 15 Apr 2019, 15:09

There is no needance in such ability in fasm syntax.
This example how hide in comment additional functionality:
that can be in 2 forms in fasm comments:

Code:

format PE GUI 4.0 ;rename binary as 'dialog.exe'
entry start       ;launch CMD line '%OUTPUT%'

1 - started with phrase ";rename binary as" to rename(not move) output
2 - started with phrase "launch CMD line" - there could be any command line of cmd.exe
in my example additional functionality is only batch scripts and they all runs in postassembly stage if it success.
; in example script(comment part) lines and fasm lines mixed, but they can be separated if needed for readability.

Thou can make addition functionality based on any script language, and launch thour own specific script wraper over fasm or fasmg.

And that will not break fasm paradigm - everyone who launch thour sources with usual fasm or fasmg will get expected actions only allowed by fasm/fasmg core. Every who will want to get additional functionality from thour sources have to launch sources via script outer to fasm|fasmg.

and even so:

Code:

;launch CMD line format c:

there is no limit what command for cmd.exe thou will use, only one rull it must fit one line.
in my variant it must fit one line.

All in thour hands.

Description:	place in same folder as fasmg and launch init bat (works only for win)	Download
Filename:	FASM.zip
Filesize:	3.82 KB
Downloaded:	833 Time(s)

15 Apr 2019, 15:09

edfed

Joined: 20 Feb 2006
Posts: 4333
Location: Now

edfed 16 Apr 2019, 07:49

Quote:

With a tool like assembler nobody expects that compiling a source could spawn files in arbitrary locations or - even worse - execute any shell commands.

i'll give a try to this cause i don't know for sure what a programmer, like me, expects from the ultimate assembler tool.

but i am ok about the shell commands, it is not really good to let the programmer execute them from the asm source cause it's platform dependent, and can lead to very specific behaviors not so portable.

"FasmShell commands" with a limited set of functionnalities would be a better choice.

the first problem i can see in this functionality is the circular compile that can occurs if a.asm compile b.asm that compiles a.asm, but it can be avoided by a sort of "include guard" mecanism.

fasm still does a lot of thing transparently, like linking, create ERROR(folders and) files, i don't see for now the problem to let it compile a bunch of binaries during the assembly of the main program.

i think it can be interresting to give a try to this idea.

about the commands in comments... it can be interresting, i'll give a try to it also and compare both solutions.

Last edited by edfed on 09 May 2019, 10:45; edited 2 times in total

16 Apr 2019, 07:49

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8353
Location: Kraków, Poland

Tomasz Grysztar 16 Apr 2019, 10:02

edfed wrote:

fasm still does a lot of thing transparently, like linking, create folders and files, i don't see for now the problem to let it compile a bunch of binaries during the assembly of the main program.

No, fasm never creates any folders, it does not even have an access to such service in its OS abstraction layer. And it only creates files in the folder it is told to (from the command line, by default the folder containing source). Read the older discussions to see what issues are there to consider.

As a related example - even functions that are quite normal for an assembler to have, like fasm's FILE (which other assemblers often call INCBIN), can be quite dangerous in specific circumstances. One time someone made as fasm-backed online assembler and shared the link - I had to immediately contact them in private to let them know that by allowing to enter any source on the form on the website they made it possible to run commands like:

Code:

file '/etc/passwd'

and then get the file by downloading the assembled binary. Obviously they had to patch the fasm on the server immediately, to disable any file accesses (to be one the safe side, INCLUDE also needs to be disabled then, as it could leak some information in fasm's error messages).

16 Apr 2019, 10:02

edfed

Joined: 20 Feb 2006
Posts: 4333
Location: Now

edfed 16 Apr 2019, 13:32

Quote:

No, fasm never creates any folders

i talked about the folder it would create if the destination folder don't exist.
and effectivelly, it don't create non existing folders... one more think i will add to my fasm version Wink

Quote:

they made it possible to run commands like:

file '/etc/passwd'

and then get the file by downloading the assembled binary.

i didn't think about this, that's good to know this problem.
i don't want this version of fasm to be used from remote places. but now i know this "funny" problem and will not forget it if i make network fasm capability.

this problem can be an interresting functionnality also if a PC runs a very simple system with only fasm on it.

16 Apr 2019, 13:32

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20337
Location: In your JS exploiting you and your system

revolution 16 Apr 2019, 14:39

Something weird happens here. Assemble this file:

Code:

file '/dev/stdin'

And get this result:

Code:

flat assembler  version 1.73.08  (4037072 kilobytes memory)
error: out of memory.

And no, I didn't type 4 billion characters on the keyboard to get the error. It happens immediately.

16 Apr 2019, 14:39

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8353
Location: Kraków, Poland

Tomasz Grysztar 16 Apr 2019, 16:22

edfed wrote:

i talked about the folder it would create if the destination folder don't exist.

fasm does no such thing.

revolution wrote:

Something weird happens here. Assemble this file:
Code:
file '/dev/stdin'    
And get this result:
Code:
flat assembler  version 1.73.08  (4037072 kilobytes memory)
error: out of memory.    
And no, I didn't type 4 billion characters on the keyboard to get the error. It happens immediately.

fasm uses lseek to find out the size of file and allocates that much memory. For files that cannot be lseek-ed the returned value is negative, so fasm ends up trying to allocate too much memory. This is an invariable behavior, perhaps that's why I never cared to add a handling of this special case there.

16 Apr 2019, 16:22

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20337
Location: In your JS exploiting you and your system

revolution 16 Apr 2019, 16:27

This code:

Code:

file '/dev/urandom'

Assembles and produces a zero byte output file.

16 Apr 2019, 16:27

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8353
Location: Kraków, Poland

Tomasz Grysztar 16 Apr 2019, 16:50

revolution wrote:

This code:
Code:
file '/dev/urandom'    
Assembles and produces a zero byte output file.

It's not my fault that system allows to seek on /dev/random with no error. Wink

16 Apr 2019, 16:50

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20337
Location: In your JS exploiting you and your system

revolution 17 Apr 2019, 00:26

I wasn't blaming you.

Thankfully this generates an error:

Code:

file 'rm -rf --no-preserve-root /'

17 Apr 2019, 00:26

edfed

Joined: 20 Feb 2006
Posts: 4333
Location: Now

edfed 17 Apr 2019, 08:07

revolution wrote:

Something weird happens here. Assemble this file:
Code:
file '/dev/stdin'    

with this line, if it worked, the ssso principle is broken due to the input of the user. hem... with anything including external files also...

the main reason why i want to add build/shell commands capabilities to fasm is to add the tedious build steps in the source, with the possibility to use fasm syntax, assembly code and powerfull word processing to make projects from a single tool.

for example a project as below:

project/main.asm
project/class1.inc
project/deps/class2.inc
project/test/main.asm

here should go the resulting executable with everything needed to run it.

project/bin/exe/main.exe
project/bin/test/main.exe
project/data/test1.bin;test2.bin

the way to achieve that, without using any external tool than fasm would be in main.asm:

Code:

compile "bin/exe/main.exe" ;compile the current file somewhere

testData1 equ "data/test1.bin"
include "class1.inc"
include "deps/class2.inc"


compile "test/main.asm" "bin/test/main.exe"

compile  testData1
{
 db 0,1,2,8
}

compile "data/test2.bin"
{
 mov eax,1
}


main:
   mov eax,0
   ret

compile ;compile the content at current directory
{
 file "bin/exe/main.exe" ;it is equivalent to copy the file compiled
}

here the compile keyword is the functionnality i talk about.

i think this functionnality is really complex to implement, but not so impossible.

_________________
Smile

fool gitlab fasmstuff
gitlab foolstuff

17 Apr 2019, 08:07

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8353
Location: Kraków, Poland

Tomasz Grysztar 17 Apr 2019, 09:54

Well, I'm saying this with a tongue in cheek, but with fasmg you can assemble multiple projects at once and encapsulate them in a ZIP file:

Code:

format binary as 'zip'
macro format?.binary? as
end macro

macro Enscapsulate Namespace*, Source*
        namespace Namespace
                macro postpone?
                        macro end?.postpone?!
                                postponed
                                esc end macro
                                purge end?.postpone?
                        end macro
                        esc macro postponed
                end macro
                macro postpone? arg
                        match ?, arg
                                macro postpone?
                                        purge postpone?
                                        macro end?.postpone?!
                                                esc end macro
                                                purge end?.postpone?
                                        end macro
                                        esc macro __postponed
                                        __postponed
                                end macro
                        end match
                        postpone
                end macro
                macro postponed
                end macro
                macro __postponed
                end macro
                include Source
                postponed
                __postponed
        end namespace
end macro

struc Assemble file
        Enscapsulate ., file
        load . : $%% from : 0
        restartout 0
end struc

;-------------------------------------------------------------------------------

DialogExample Assemble 'dialog.asm'

Template Assemble 'template.asm'

;-------------------------------------------------------------------------------

include 'zip.inc'

zipfile 'readme.txt'

        db 'Example of multiple binaries generated in a single fasmg execution.'

zipfile 'bin/dialog.exe'

        db DialogExample

zipfile 'bin/template.exe'

        db Template

This demo uses zip.inc and some examples from my GitHub repository.

17 Apr 2019, 09:54

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20337
Location: In your JS exploiting you and your system

revolution 17 Apr 2019, 10:32

edfed wrote:

the main reason why i want to add build/shell commands capabilities to fasm is to add the tedious build steps in the source, with the possibility to use fasm syntax, assembly code and powerfull word processing to make projects from a single tool.

If you were to consider a scripting language (batch files for example, or maybe in IDE) as your primary tool then you can use fasm as the assembler, fossil as the SCM, 7-zip as the compressor, curl as the uploader, etc.

It is just a matter of thinking of fasm as one of the set of tools you use to manage, create, advertise and distribute your programs.

17 Apr 2019, 10:32

rugxulo

Joined: 09 Aug 2005
Posts: 2341
Location: Usono (aka, USA)

rugxulo 18 Apr 2019, 05:41

Sounds like you want something like WEB (literate programming) or SHAR (shell archives). Or maybe you just want some portable Make variant. (GNU Make is fairly common, even FPC includes it, even on Win32. There's also OpenWatcom's WMake and DMake and others. Those are all mostly portable, depending on what you're trying to do.)

18 Apr 2019, 05:41

guignol

Joined: 06 Dec 2008
Posts: 763

guignol 21 Apr 2019, 08:53

fasm should have had deflate support a long gime ago

and FASMD access to fasm board, too

21 Apr 2019, 08:53

guignol

Joined: 06 Dec 2008
Posts: 763

guignol 24 Apr 2019, 14:59

What is wrong with having a zipped UTF-32 as a .fasm source file?

24 Apr 2019, 14:59

edfed

Joined: 20 Feb 2006
Posts: 4333
Location: Now

edfed 27 Apr 2019, 09:05

Quote:

fasm should have had deflate support a long gime ago

and FASMD access to fasm board, too

i think it is not the same functionnality.

in my case, i can compile the .dll used by my testing .exe from the source.
in your case, i just feel you want to do some absurd examples to explain me how my idea is ridiculous...

no problem with that, as always, when nobody agree with me, i do alone.

27 Apr 2019, 09:05

guignol

Joined: 06 Dec 2008
Posts: 763

guignol 27 Apr 2019, 17:20

look, Tomasz could develop his own LZ77-based compression method

anyway, on modern machinery still using raw text sources is ridiculous

27 Apr 2019, 17:20

edfed

Joined: 20 Feb 2006
Posts: 4333
Location: Now

edfed 27 Apr 2019, 23:09

ho, i see. yes you are true. i think sometimes about this ridiculous use of repetive text tokens to code. but it means you should continue to get a human readable source code. then, or something between binary and text, or graphix.

for my fool langage, i want to make a sort of graphic editor, letting you edit your code while running.
means i would not use the source file, but directlly use the binary + a binary parser to get the edition working.

maybe we should then just make a new kind of disassembler to do the job...

27 Apr 2019, 23:09

Goto page 1, 2 Next

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum