As a warm-up before perhaps writing a general introduction to assembly language with the assistance of fasmg I consider writing a series of small tutorials explaining various executable and object formats.

The main tool used during the tutorials would be fasmg, for its capability to generate everything we would ever need, including x86 code with relocations. Generators for various formats already exist for fasmg, but the point of the tutorials would be to write them from scratch, learning as much about the format as possible during the process.

I would like to write at least one, to see how well would it be received, and then decide whether to continue with the series. What format would you like to see explained this way?

Since PE is the most commonly used format, then this is the place to start IMO. Some of the less used parts can be safely ignored, also IMO.

MZ is "simple" of course, but many people can't easily run the 16-bit programs on their 64-bit OSes.

I can`t say anything against PE as first one. So maybe elf be second (in case of elf: libraries are more interest then executables itself). And Mach-O third.

Other: .sys (device driver)

Device drivers nowadays use PE format, nothing else.

Well, I could also put LE (which was used for Win9x device drivers among other things) on the list next to OMF. Both are mostly irrelevant today.

Isn't MZ / COFF part of PE structure? You can''t explain one without involving the other(s). I mean there's a COFF header somewhere in a PE format.

You should definitely take a look some of my recent tweets.

I am already midway through with writing the first chapter of the tutorials, about PE. I use a dummy stub, so I do not need to explain much about MZ. In fact, I could probably get away with explaining next to nothing, but to keep up appearances I at least set up the stub length fields properly, so the stub appears valid even though it has no actual code.

As for the COFF, of course I mention it, as it is the main source of some of strange and/or obsolete values, but PE really does use only a few of the COFF's structures and the linkable object in COFF format is very, very different from an executable PE image. Things like relocations are done completely differently - obviously for linkable object relocations are much more complex. PE does not use a symbol table, which is essential for object COFF, etc. etc. Therefore COFF by itself deserves another chapter. Though I think that when I'm finished with PE I'm going to go for ELF first.

It will be a useful read indeed. I am eager to know how, for example, one can change the structure of a PE format and still get away with it without the OS complaining.

For example, GoLink managed to put the "PE" signature at offset 0x60 rather than the usual 0x80 as specified.

Interesting.

The example constructed in my tutorial has it on offset 0x40.

If the moving offset of PE looked strange to you, please try this: take the source code of FASMW and replace the first line (FORMAT directive) with: Before assembling it you need to have assembled FASMD.ASM in its directory, too.
What you get is a dual executable, it should show up as FASMD when run in DOS and as FASMW when run in Windows.

Looking forward to read whatever will be the result. Tweet about the source of 0x10B value is the kind of things I love most in such texts: the way history shines through modern abstractions and formats.

You and a few others here may be interested in some related work I did in this area: https://toddcullumresearch.com/2017/07/16/portable-executable-file-corruption/

I think a good tutorial / tutorials about FASM can be start with this title:
"How to manage a big project with FASM" in order to have/see the flow of good ways of programming with FASM.

I have the direction established already, and it is one I'm quite satisfied with. There is so much to say and it is all about things that I really wanted to write about.

Also, the order of "PE, then ELF" turned out to be an excellent choice, it allows me to introduce features of fasmg quite slowly (my hope is to not intimidate with them if a reader happens to be interested more in the experimentation with binary formats than in fasmg).

I impressed that "#" used so rarely and in most cases dot used for concatenation tokens.
Looks like tutorial about PE for 32bit address machines (no matter 32bit or 64bit they are, alpha was 64bit on PE32, and PE everywhere PE) is done, stayed PE version for machines with 64bit address.
--------------------------------------------------------------------------------------
To catafest: would thou like to start such tutorial thourself?
Any big project is constructor - part 1 project management: thou have to split it on related logical bloccks, some blocks could be portable, some not. And some of such blocks could be already created by other people (includes or external libraries). Than thou can build skelet of thour future program - working executable in thour target OS.
part 2 process management - thou have to determine in what stage thour application could be - with every stage related some piece of data process operated on and data that stayed unchanged for current stage.
part 3 data and memory management - thou have to determine what structures thou needed, how they will be related, how they will be stored (static, stack, heap, registers, or etc.) in stages that thou determine earlier.
something like that...

start thour big project and share ideas or experience on the fly.
What thou want to create? If it will be interest people can join to thou.

I still plan to at least touch the subject of resources before I go to PE+.

Hi,

To my best knowledge there's no such thing. I've written a bootloader for long mode kernels in PE format, and both x86_64 and AArch64 PE/COFF had 32 bit addresses which I had to sign-extend manually to 64 bit addresses (keeping the sign required for higher half only). It was just the code segment that had 64 bit instructions, that's all. I believe that's the reason why the format is called PE32+ and not PE64.

Cheers,
bzt

PE+ as a format is able to handle 64-bit VAs fine. Only the RVAs are still limited to 32 bits.

I have already written about this in the tutorial, too.

any file format is ok, i am sure there are common factors between them