flat assembler
Message board for the users of flat assembler.

Index > Programming Language Design > 386 assembly with fasm g

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 21 Oct 2015, 21:20
I have created the set of macroinstructions for fasm g that implement the 80386 instruction set. What I had in mind was to try making fasm g self-hosting, even though it would be completely unpractical because of how slow it is to assemble large project this way. But it is an interesting demonstration of the capabilities of fasm g macro engine.

In my preliminary tests that terribly slow self-assembly took almost a minute on my current machine. Coincidentally, this is the same time that the first self-assembly of fasm took on my oldest computer back in 1999 (well, later fasm got much better). This could be seen as a manifestation of Wirth's law. Not that I envision this to be the right way of making assemblers now. Wink But there was something wrong in this case, because it took way too many passes to resolve its own source - it had no such problem on the other sources I tried. There is probably some hidden bug that caused these excess passes and when I fix it, the self-assembly time may become a bit more manageable.

The full demonstration of fasm g self-hosting requires that I also create macros to generate PE/ELF output - that should not be too hard but it remains to be done. For now I combined the new macros with the ones for MZ executable creation and various DOS samples can be assembled this way, including the DOS version of fasm 1 and the kelvar example. I am attaching the latter here, packaged with the new 386 macros used to assemble it.

On a side note, I replaced the original kelvar package in the Examples section with a new one, because it still contained the old version of unREAL engine. The new one checks whether processor supports 32-bit real mode and shows an appropriate error otherwise - it is important in the era of emulators, because many of them (including DOSBox) do not emulate 32-bit real mode properly. I also changed the music file, because the one I shipped with kelvar originally was a down-sampled song by a known artist and I'm not sure whether that could be qualified as a fair use. I replaced it with my own recording that has no such problems because I have complete rights to it.


Description: kelvar example assembled with fasm g
Download
Filename: kelvar_fasmg.zip
Filesize: 1.83 MB
Downloaded: 1160 Time(s)

Post 21 Oct 2015, 21:20
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 22 Oct 2015, 14:19
I have found out the reason why fasm g took so many passes while assembling itself while it had no such problem when assembling fasm 1 sources - the "shift prediction" heuristic was not implemented fully in the same way as it was in fasm 1. After I corrected this, the self-assembly time of fasm g is down to 8 seconds on my machine. Now that is something almost bearable. Smile
Post 22 Oct 2015, 14:19
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20526
Location: In your JS exploiting you and your system
revolution 22 Oct 2015, 14:28
I couldn't find the 386 macros in any of the downloads. I assume they are still a work in progress?
Post 22 Oct 2015, 14:28
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 22 Oct 2015, 15:32
Yes, they are not complete, though are enough to assemble the most of my 32-bit things. You can find the current version in the attachment in my initial post above. I packaged kelvar with fasmg and the 80386 and MZ macros, so that it is all ready to run.
Post 22 Oct 2015, 15:32
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20526
Location: In your JS exploiting you and your system
revolution 22 Oct 2015, 16:27
Found them. Can we assume it will make it to the repo and the download page once they are all debugged and completed?

I think it would be great if users can select their target CPU by simply including the appropriate macro set. Anything from 8086 up to the latest i7, with all the CPUs in between, as separate files for each.
Post 22 Oct 2015, 16:27
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 22 Oct 2015, 16:28
Hi, Tomasz.

I have several questions.

1. I like the new macro syntax and the nested macros without escaping (I want to have it in FASM1 as well, but dreams, dreams Wink ). But how to make nested macros that start in one macro and end in another? Something like:
Code:
macro global
  macro MakeGlobal
end macro

macro endg
  end macro
end macro

global
  var2 db "B"
  var3 db "C"
endg

var1 db "A"

MakeGlobal    

The above code should generate "ABC", but it makes "BCA"

2. Are you plan to keep this way of code generation for FASM 2?
Post 22 Oct 2015, 16:28
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 22 Oct 2015, 17:07
JohnFound wrote:
1. I like the new macro syntax and the nested macros without escaping (I want to have it in FASM1 as well, but dreams, dreams Wink ). But how to make nested macros that start in one macro and end in another? ...
It should go like this:
Code:
macro global
  esc macro MakeGlobal
end macro

macro endg!
  esc end macro
end macro

global
  var2 db "B"
  var3 db "C"
endg

var1 db "A"

MakeGlobal    
The ESC directive allows to bypass the proper nesting checks, and the unconditional macro (defined with "!") takes the role FIX directive has in analogous constructs for fasm 1.

JohnFound wrote:
2. Are you plan to keep this way of code generation for FASM 2
I'm not sure what exactly do you mean. I distinguished fasm g from fasm 2, because fasm g does not implement any instruction encoder internally, while fasm 2 was planned to have one for x86/x64 (and I wanted it to be a more sophisticated one compared to fasm 1). But I don't whether I can make actual fasm 2 any time soon.
Post 22 Oct 2015, 17:07
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 22 Oct 2015, 17:52
Tomasz Grysztar wrote:
It should go like this:


Oh, I see. Very elegant syntax!

_________________
Tox ID: 48C0321ADDB2FE5F644BB5E3D58B0D58C35E5BCBC81D7CD333633FEDF1047914A534256478D9
Post 22 Oct 2015, 17:52
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 23 Oct 2015, 10:26
revolution wrote:
Found them. Can we assume it will make it to the repo and the download page once they are all debugged and completed?

I think it would be great if users can select their target CPU by simply including the appropriate macro set. Anything from 8086 up to the latest i7, with all the CPUs in between, as separate files for each.
Yes, the basic package already contains 8086.inc and 80186.inc and I plan to include 80386.inc when it's ready. I skipped 286, but it should be easy to make one once we have 386 ready. As for the macros for later processors, I had no serious plans for them so far.
Post 23 Oct 2015, 10:26
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 23 Oct 2015, 17:37
Another sample of assembling existing source without any changes other than including the macro sets - this time it is my TetrOS. I had to include another macro that adds support for anonymous labels, because fasm g does not have them by itself. The package contains the latest version of my 80386 macros.

Also, if you are interested how I tested the self-assembly of fasm g, I had to add a "struct" macro there, and it looks like this:
Code:
macro struct? name
        macro ends?!
                        end namespace
                end struc
                virtual at 0
                        name name
                        sizeof.name = $
                end virtual
        end macro
        struc name
                label . : sizeof.name
                namespace .
end macro    
It it not an implementation comparable to the one in standard headers for fasm (that would be lot more complex, and it would probably be the first time I actually used EVAL directive I implemented into fasm g), but it is more than enough to cover all the use cases in fasm g sources.


Description: TetrOS source adapted to fasm g
Download
Filename: TetrOS.zip
Filesize: 8.88 KB
Downloaded: 1126 Time(s)

Post 23 Oct 2015, 17:37
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 25 Oct 2015, 21:42
I have added 286 and 386 macros to the official fasm g package, together with the recycled usedpmi.asm example that came with fasm 1 for DOS. The next step would be to create PE/ELF formatting macros and then make fasm g self-hosting (I would still leave the standard source requiring fasm 1, I am not that insane, but I would include the self-hosting variant because I think it's kind of cool). That will require another batch of my spare time and I don't when that's going to be. But when I have that ready, my following plans are to finally add basic support for floats, and then make sets of x87 macros and copy the FPU example from fasm 1 package - this would complete my starting road map for fasm g and it could be marked as 1.0 release. Then I would go on to create more documentation/tutorials and work on the "assembler creation kit" based on fasm g, in a hope that this would help others to make some use of it.
Post 25 Oct 2015, 21:42
View user's profile Send private message Visit poster's website Reply with quote
idle



Joined: 06 Jan 2011
Posts: 440
Location: Ukraine
idle 26 Oct 2015, 09:58
I'm glad fasmg macro makes implementing of new features pleasant and clear for both you the author and other source readers.

Tomasz, will fasmg be capable to compile fasm 1.71.xx files, e.g. include "syntax/fasm1.inc"?
Post 26 Oct 2015, 09:58
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 26 Oct 2015, 13:26
idle wrote:
Tomasz, will fasmg be capable to compile fasm 1.71.xx files, e.g. include "syntax/fasm1.inc"?
As the above samples show, many of the sources written for fasm 1 can be adapted with little changes. But if you mean the macro-based emulation of entire fasm 1 syntax, including features of preprocessor, then my answer is that I don't think it would be a good idea. It is possible in theory, thanks to features like EVAL, but I suspect that it would be highly unpleasant.
Post 26 Oct 2015, 13:26
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 29 Oct 2015, 11:01
JohnFound wrote:
Oh, I see. Very elegant syntax!
On a side note, in this sample you can even place "MakeGlobal" line on top of source - when a macro is uniquely defined (that is: exactly once), it can be forward-referenced.
Post 29 Oct 2015, 11:01
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 30 Oct 2015, 18:42
I have created the PE formatter in form of macros for fasm g, and so I'm almost ready to make fasm g self-hosting on Windows platform. The macros are mostly compatible with syntax of fasm 1, except for the "format PE" line with settings, but I may it that later.

I'm attaching current version of macros together with adapted "PE demo" that comes with fasm 1. You may notice that even the fixups can be created this way - if the MZ examples were not enough, this definitely demonstrates that it is possible to create code with relocations using just macros in fasm g.

There is also no checksum computation in this version, but this should be very easy to add.


Description: PE demo from fasm package, adapted to fasm g
Download
Filename: pedemo.zip
Filesize: 10.64 KB
Downloaded: 1058 Time(s)

Post 30 Oct 2015, 18:42
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 31 Oct 2015, 22:12
The PE macros and Win32 example are now part of the official fasm g package.

I have also prepared the complete set of macros that allow fasm g to assemble itself in its Win32 version. I still need to decide how to organize the files to include the self-hosting option in the official package, but for now I attach the required files here for anyone interested. The "selfhost" folder should be placed inside "source/win32".


Description: Macros for fasm g self-assembly
Download
Filename: selfhost.zip
Filesize: 13 KB
Downloaded: 1050 Time(s)

Post 31 Oct 2015, 22:12
View user's profile Send private message Visit poster's website Reply with quote
idle



Joined: 06 Jan 2011
Posts: 440
Location: Ukraine
idle 01 Nov 2015, 12:42
That's is strong, you are cool!
Post 01 Nov 2015, 12:42
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 02 Nov 2015, 15:54
The latest fasm g package includes the self-hosting under Win32. The altered source exploits the quirks of fasm syntax to detect whether it is being assembled by fasm 1 or fasm g and include appropriate headers. The tricky part goes like this:
Code:
match ,{

        ; fasm 1 detected
        include 'win32a.inc'

} match -,{
else

        ; fasm g detected
        include 'selfhost.inc'

end match
_ equ }    
Next in the queue are ELF executable format macros and self-hosting Linux version. This one should be easier.
Post 02 Nov 2015, 15:54
View user's profile Send private message Visit poster's website Reply with quote
idle



Joined: 06 Jan 2011
Posts: 440
Location: Ukraine
idle 03 Nov 2015, 09:13
Tomasz, fasmg expected to be more flexible - do you plan to add include once, multiline comments, command line access, fasmg ide, %time.YYYY/MM/DD assembly var?
Post 03 Nov 2015, 09:13
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8367
Location: Kraków, Poland
Tomasz Grysztar 03 Nov 2015, 10:54
fasm g is a bare engine that still is enough to implement everything in form of macros, and can be therefore seen as an ultimate embodiment of my "complex solutions with simple features" idea. All the added features are the ones that make it possible to do even more things with macros only. The implementation of multi-line comments with a very simple macro is shown in the manual, various variants of "include once" (depending on what problem you look to solve) are also easy to make. I planned to add more built-in parameters but I do not want to clutter the namespace too much so I decided to wait until I see which ones I really need. I do not have plans for any new IDE and I do not know what you meant by "command line access".

What I wanted to achieve with fasm g is perhaps a demonstration of my own concept of what an assembler is (the idea I originally came up with in my "Understanding fasm" text.
Post 03 Nov 2015, 10:54
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2025, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.