flat assembler
Message board for the users of flat assembler.

Index > DOS > bootstrapping FASM?

Author
Thread Post new topic Reply to topic
rugxulo



Joined: 09 Aug 2005
Posts: 2341
Location: Usono (aka, USA)
rugxulo 20 Feb 2018, 05:08
Someone in another subforum here (re: making a FASM package for Gentoo) mentioned the idea of bootstrapping FASM.

To me, the simplest way is to use an old DOS .COM version of FASM to build a newer version and then ultimately use that to build the latest version.

So, if someone really "needed" (wanted?) to bootstrap FASM from something else (NASM?), all they'd have to do is convert the ancient .COM version and go on from there.

Of course, that's not directly useful to Gentoo, obviously, but it's not hard to setup some kind of DOS-compatible environment.

And yes, I know, totally pointless since it's obvious that FASM can (and should) continue to build itself. I'm not paranoid enough to worry about The Ken Thompson Hack. To me, it's fairly obvious that FASM does only what it implies.

Code:
@echo off
REM ... bootstrapping FASM from other asm? here's where to start ...

for %%a in (140 164 172) do unzip -q fasm%%a -d fasm%%a -x *.exe *.com *.sys

unzip -qj fasm140 SOURCE\DOS\FASM.COM

cd fasm164\source\dos
gsed -i -e "1i\salc equ setalc" ..\formats.inc
gsed -i -e "1i\macro align value{rb(value-1)-($+value-1)mod value}" fasm.asm
..\..\..\fasm.com fasm.asm f1.exe >NUL
f1.exe fasm.asm f2.exe >NUL
echo.
echo (FASM 1.64)
unzip -qqv ..\..\..\fasm164.zip fasm.exe | awk "{print $8,toupper($7)}"
crc32 f2.exe
REM 5DAC0852
echo.

cd ..\..\..\fasm172\source\dos
echo short equ byte>oldshort.inc
gsed -i -e "1i\include 'oldshort.inc'" modes.inc
..\..\..\fasm164\source\dos\f2.exe fasm.asm f3.exe >NUL
echo. >oldshort.inc
f3.exe fasm.asm good.exe >NUL
echo.
echo (FASM 1.72)
unzip -qqv ..\..\..\fasm172.zip fasm.exe | awk "{print $8,toupper($7)}"
crc32 good.exe
REM 5F7C1ECC
echo.

cd ..\..\..
rm -rf fasm140 fasm164 fasm172
rm -f fasm.com
    
Post 20 Feb 2018, 05:08
View user's profile Send private message Visit poster's website Reply with quote
Furs



Joined: 04 Mar 2016
Posts: 2543
Furs 20 Feb 2018, 13:10
If someone was paranoid then yeah, this actually makes sense, surely DOSBox is available in Gentoo as well. (I honestly don't care that much about "official distro repositories" though)
Post 20 Feb 2018, 13:10
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20416
Location: In your JS exploiting you and your system
revolution 21 Feb 2018, 06:20
It is actually quite easy to disassemble the binary and compare to the sources. And if you are really good with awk/sed/etc then it could even be automated. That might make for an interesting project for someone to make an automated verification tool.
Post 21 Feb 2018, 06:20
View user's profile Send private message Visit poster's website Reply with quote
rugxulo



Joined: 09 Aug 2005
Posts: 2341
Location: Usono (aka, USA)
rugxulo 22 Feb 2018, 00:03
revolution wrote:
It is actually quite easy to disassemble the binary and compare to the sources.


You mean .COM? Obviously that's about as simple as you can get, (almost) raw binary. That was my point, it's somewhat easier than using ELF and a linker.

But various potential complications can occur (self-modifying code, cpu-specific code, data put between code instead of only in one place, etc). So disassembly isn't always simple.

Also, x86 is variable-length, which complicates things. So if you change one instruction, all the other displacements get adjusted accordingly. So it's hard to disassemble two similar binaries, even if they only differ in length of common JMPs (two bytes or three?).

Maybe comparing (partial) listing would suffice? Nope, not if it doesn't simplify / unify / normalize basic forms, not counting other complex stuff.

revolution wrote:

And if you are really good with awk/sed/etc then it could even be automated. That might make for an interesting project for someone to make an automated verification tool.


In what way? I did make a simple script to compare PSR Invaders .COM, using NDISASM + AWK to avoid the raw opcodes and only compare actual (100% equivalent) instructions. "mov cx,ax" (8B C8) vs. "mov cx,ax" (89 C1), etc.

Of course, FASM and NASM share most opcodes, so that's less of a problem here. They are usually identical output. (I did sometimes find one rare quirk difference in YASM, though.)

I don't think anyone here really wants me to convert it to NASM for us. But it would be an interesting, albeit pointless, task.
Post 22 Feb 2018, 00:03
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20416
Location: In your JS exploiting you and your system
revolution 22 Feb 2018, 08:50
Compare by disassembled instructions, that is, the text output from the disassembler compared to the text input from the source. It really isn't that hard to do if one has the need. But if there is no need then it is just either a waste of time or a learning opportunity for someone. For a one-off job then comparing by sight will be a bit tedious but quite doable.

Even a really simple instruction count comparison can give a lot of confidence that there are no unexpected bad things in there. It would be really really hard to keep the instruction count the same but at the same time have it altered in some way to make it do bad things. In fact so hard that I would almost discount it as not worth worrying about unless you have some extra special need where a more thorough investigation is warranted. Or if you are bored one day.

Note that my definition of "bad things" above covers a lot of situations. Like deliberate meddling, bad encoding, corrupted bits, missing instructions, duplicated instructions, etc.
Post 22 Feb 2018, 08:50
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8356
Location: Kraków, Poland
Tomasz Grysztar 22 Feb 2018, 13:13
You could use fasmg to bootstrap fasm, even providing your own copy of x86 macros. Since everything is a macro then, the instruction handlers could be easily modified so that they would collect additional statistics and/or checksums that could then be used to verify the generated file.
Post 22 Feb 2018, 13:13
View user's profile Send private message Visit poster's website Reply with quote
rugxulo



Joined: 09 Aug 2005
Posts: 2341
Location: Usono (aka, USA)
rugxulo 21 Aug 2018, 00:13
rugxulo wrote:
Someone in another subforum here ... mentioned the idea of bootstrapping FASM.

To me, the simplest way is to use an old DOS .COM version of FASM to build a newer version and then ultimately use that to build the latest version.


Gotta love how I unzipped fasm140.zip, then deleted it, but never used it!

The main complaint that I had here, looking back, was using GNU sed. I didn't think it was necessary. Shouldn't FASM be able to handle "salc equ setalc", even if appended at the very end of the source file? Surely preprocessing is done first, no? Ugh, apparently not. (Or at least not in old 1.40. But maybe I'm misunderstanding ... probably!)

So, basically, you don't need GNU sed, just COMMAND.COM's "COPY /B +" (and ECHO redirecting to file). Also, I just used DELTREE since that's relatively common (since something like DOS v5 ??) instead of relying on *nix RM.

So yeah, probably not worth mentioning, but I'm doing so anyways.

Quote:

@echo off
REM ... bootstrapping FASM from other asm? here's where to start ...

for %%a in (164 172) do unzip -q fasm%%a -d fasm%%a -x *.exe *.com *.sys

unzip -qnj fasm140 SOURCE\DOS\FASM.COM

cd fasm164\source\dos
ren ..\formats.inc *.in~
echo salc equ setalc >..\formats.in2
copy /b ..\formats.in2 + ..\formats.in~ ..\formats.inc >NUL

ren fasm.asm *.as~
echo macro align value{rb(value-1)-($+value-1)mod value} >fasm.as2
copy /b fasm.as2 + fasm.as~ fasm.asm >NUL
..\..\..\fasm.com fasm.asm f1.exe >NUL
f1.exe fasm.asm f2.exe >NUL
echo.
echo (FASM 1.64)
unzip -qqv ..\..\..\fasm164.zip fasm.exe | awk "{print $8,toupper($7)}"
crc32 f2.exe | awk "{printf(\"%%-8s %%s\n\",$1,$2)}"
REM 5DAC0852
echo.

cd ..\..\..\fasm172\source\dos
echo short equ byte >oldshort.inc
echo include 'oldshort.inc' >modes.in2
ren modes.inc *.in~
copy /b modes.in2 + modes.in~ modes.inc >NUL
..\..\..\fasm164\source\dos\f2.exe fasm.asm f3.exe >NUL
echo. >oldshort.inc
f3.exe fasm.asm good.exe >NUL
echo.
echo (FASM 1.72)
unzip -qqv ..\..\..\fasm172.zip fasm.exe | awk "{print $8,toupper($7)}"
crc32 good.exe | awk "{print $1,$2}"
REM 5F7C1ECC
echo.

cd ..\..\..
ctty nul
for %%z in (fasm164 fasm172) do call deltree /y %%z
del fasm.com
ctty con
Post 21 Aug 2018, 00:13
View user's profile Send private message Visit poster's website Reply with quote
pts



Joined: 25 Nov 2024
Posts: 1
pts 25 Nov 2024, 03:53
I was able to bootstrap the latest fasm (1.73.32, released on 2023-12-24) from its source and fasm 1.20 (released on 2001-11-17) source (and, alternatively, fasm 1.37 source) for Linux i386 host only. For that I wrote a Perl script with a few dozen regexp substitutions to transform fasm 1.20 source from fasm syntax to NASM syntax, and compile it with NASM >=0.95. With the fasm 1.20 executable program, I was able to compile the fasm 1.73.32 source. I had to patch the fasm 1.73.32 source a little bit, because fasm 1.20 wasn't able to add the ELF-32 headers automatically, and it had some directives missing. More details here: https://github.com/pts/pts-fasm-bootstrap

With this method it would be easy to bootstrap any version of fasm between versions 1.20 and 1.73.32, for any host system: first bootstrap for Linux i386 host, and then compile for the other supported hosts using the Linux i386 executable program.

Going back to versions earlier than 1.20 looks like possible. fasm 1.04 (released on 2000-08-10) was the earliest which was able to create Win32 executable programs. For any version earlier than 1.37, the Linux host support (SOURCE/LINUX/FASM.ASM and SOURCE/LINUX/SYSTEM.INC) has to be manually backported from fasm 1.37. I've done this for fasm 1.20 (I had to rename some registers because of function call ABI changes), and it can be done for other, earlier versions as well.
Post 25 Nov 2024, 03:53
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20416
Location: In your JS exploiting you and your system
revolution 26 Nov 2024, 12:41
There is also another topic that shows how to bootstrap fasm in Windows, DOS and Linux from nothing all the way to the current version.

https://board.flatassembler.net/topic.php?t=22633
Post 26 Nov 2024, 12:41
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.