flat assembler
Message board for the users of flat assembler.
 Home   FAQ   Search   Register 
 Profile   Log in to check your private messages   Log in 
flat assembler > Programming Language Design > 386 assembly with fasm g

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland
386 assembly with fasm g
I have created the set of macroinstructions for fasm g that implement the 80386 instruction set. What I had in mind was to try making fasm g self-hosting, even though it would be completely unpractical because of how slow it is to assemble large project this way. But it is an interesting demonstration of the capabilities of fasm g macro engine.

In my preliminary tests that terribly slow self-assembly took almost a minute on my current machine. Coincidentally, this is the same time that the first self-assembly of fasm took on my oldest computer back in 1999 (well, later fasm got much better). This could be seen as a manifestation of Wirth's law. Not that I envision this to be the right way of making assemblers now. Wink But there was something wrong in this case, because it took way too many passes to resolve its own source - it had no such problem on the other sources I tried. There is probably some hidden bug that caused these excess passes and when I fix it, the self-assembly time may become a bit more manageable.

The full demonstration of fasm g self-hosting requires that I also create macros to generate PE/ELF output - that should not be too hard but it remains to be done. For now I combined the new macros with the ones for MZ executable creation and various DOS samples can be assembled this way, including the DOS version of fasm 1 and the kelvar example. I am attaching the latter here, packaged with the new 386 macros used to assemble it.

On a side note, I replaced the original kelvar package in the Examples section with a new one, because it still contained the old version of unREAL engine. The new one checks whether processor supports 32-bit real mode and shows an appropriate error otherwise - it is important in the era of emulators, because many of them (including DOSBox) do not emulate 32-bit real mode properly. I also changed the music file, because the one I shipped with kelvar originally was a down-sampled song by a known artist and I'm not sure whether that could be qualified as a fair use. I replaced it with my own recording that has no such problems because I have complete rights to it.


Description: kelvar example assembled with fasm g
Download
Filename: kelvar_fasmg.zip
Filesize: 1.83 MB
Downloaded: 66 Time(s)

Post 21 Oct 2015, 21:20
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland
I have found out the reason why fasm g took so many passes while assembling itself while it had no such problem when assembling fasm 1 sources - the "shift prediction" heuristic was not implemented fully in the same way as it was in fasm 1. After I corrected this, the self-assembly time of fasm g is down to 8 seconds on my machine. Now that is something almost bearable. Smile
Post 22 Oct 2015, 14:19
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 14794
Location: Lost in translation
I couldn't find the 386 macros in any of the downloads. I assume they are still a work in progress?
Post 22 Oct 2015, 14:28
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland
Yes, they are not complete, though are enough to assemble the most of my 32-bit things. You can find the current version in the attachment in my initial post above. I packaged kelvar with fasmg and the 80386 and MZ macros, so that it is all ready to run.
Post 22 Oct 2015, 15:32
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 14794
Location: Lost in translation
Found them. Can we assume it will make it to the repo and the download page once they are all debugged and completed?

I think it would be great if users can select their target CPU by simply including the appropriate macro set. Anything from 8086 up to the latest i7, with all the CPUs in between, as separate files for each.
Post 22 Oct 2015, 16:27
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3434
Location: Bulgaria
Hi, Tomasz.

I have several questions.

1. I like the new macro syntax and the nested macros without escaping (I want to have it in FASM1 as well, but dreams, dreams Wink ). But how to make nested macros that start in one macro and end in another? Something like:

Code:
macro global
  macro MakeGlobal
end macro

macro endg
  end macro
end macro

global
  var2 db "B"
  var3 db "C"
endg

var1 db "A"

MakeGlobal


The above code should generate "ABC", but it makes "BCA"

2. Are you plan to keep this way of code generation for FASM 2?
Post 22 Oct 2015, 16:28
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland

JohnFound wrote:
1. I like the new macro syntax and the nested macros without escaping (I want to have it in FASM1 as well, but dreams, dreams Wink ). But how to make nested macros that start in one macro and end in another? ...

It should go like this:

Code:
macro global
  esc macro MakeGlobal
end macro

macro endg!
  esc end macro
end macro

global
  var2 db "B"
  var3 db "C"
endg

var1 db "A"

MakeGlobal

The ESC directive allows to bypass the proper nesting checks, and the unconditional macro (defined with "!") takes the role FIX directive has in analogous constructs for fasm 1.


JohnFound wrote:
2. Are you plan to keep this way of code generation for FASM 2

I'm not sure what exactly do you mean. I distinguished fasm g from fasm 2, because fasm g does not implement any instruction encoder internally, while fasm 2 was planned to have one for x86/x64 (and I wanted it to be a more sophisticated one compared to fasm 1). But I don't whether I can make actual fasm 2 any time soon.
Post 22 Oct 2015, 17:07
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3434
Location: Bulgaria

Tomasz Grysztar wrote:
It should go like this:



Oh, I see. Very elegant syntax!

_________________
Tox ID: A48DEF727DF44C3B5C2E576B65021F1A45D8FA52E2F8E257F1CAE148BBADB162FDF7820BD1F9
Post 22 Oct 2015, 17:52
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland

revolution wrote:
Found them. Can we assume it will make it to the repo and the download page once they are all debugged and completed?

I think it would be great if users can select their target CPU by simply including the appropriate macro set. Anything from 8086 up to the latest i7, with all the CPUs in between, as separate files for each.

Yes, the basic package already contains 8086.inc and 80186.inc and I plan to include 80386.inc when it's ready. I skipped 286, but it should be easy to make one once we have 386 ready. As for the macros for later processors, I had no serious plans for them so far.
Post 23 Oct 2015, 10:26
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland
Another sample of assembling existing source without any changes other than including the macro sets - this time it is my TetrOS. I had to include another macro that adds support for anonymous labels, because fasm g does not have them by itself. The package contains the latest version of my 80386 macros.

Also, if you are interested how I tested the self-assembly of fasm g, I had to add a "struct" macro there, and it looks like this:

Code:
macro struct? name
        macro ends?!
                        end namespace
                end struc
                virtual at 0
                        name name
                        sizeof.name = $
                end virtual
        end macro
        struc name
                label . : sizeof.name
                namespace .
end macro

It it not an implementation comparable to the one in standard headers for fasm (that would be lot more complex, and it would probably be the first time I actually used EVAL directive I implemented into fasm g), but it is more than enough to cover all the use cases in fasm g sources.


Description: TetrOS source adapted to fasm g
Download
Filename: tetros.zip
Filesize: 8.53 KB
Downloaded: 54 Time(s)

Post 23 Oct 2015, 17:37
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland
I have added 286 and 386 macros to the official fasm g package, together with the recycled usedpmi.asm example that came with fasm 1 for DOS. The next step would be to create PE/ELF formatting macros and then make fasm g self-hosting (I would still leave the standard source requiring fasm 1, I am not that insane, but I would include the self-hosting variant because I think it's kind of cool). That will require another batch of my spare time and I don't when that's going to be. But when I have that ready, my following plans are to finally add basic support for floats, and then make sets of x87 macros and copy the FPU example from fasm 1 package - this would complete my starting road map for fasm g and it could be marked as 1.0 release. Then I would go on to create more documentation/tutorials and work on the "assembler creation kit" based on fasm g, in a hope that this would help others to make some use of it.
Post 25 Oct 2015, 21:42
View user's profile Send private message Visit poster's website Reply with quote
idle



Joined: 06 Jan 2011
Posts: 340
Location: ukraina
I'm glad fasmg macro makes implementing of new features pleasant and clear for both you the author and other source readers.

Tomasz, will fasmg be capable to compile fasm 1.71.xx files, e.g. include "syntax/fasm1.inc"?
Post 26 Oct 2015, 09:58
View user's profile Send private message Send e-mail Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland

idle wrote:
Tomasz, will fasmg be capable to compile fasm 1.71.xx files, e.g. include "syntax/fasm1.inc"?

As the above samples show, many of the sources written for fasm 1 can be adapted with little changes. But if you mean the macro-based emulation of entire fasm 1 syntax, including features of preprocessor, then my answer is that I don't think it would be a good idea. It is possible in theory, thanks to features like EVAL, but I suspect that it would be highly unpleasant.
Post 26 Oct 2015, 13:26
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland

JohnFound wrote:
Oh, I see. Very elegant syntax!

On a side note, in this sample you can even place "MakeGlobal" line on top of source - when a macro is uniquely defined (that is: exactly once), it can be forward-referenced.
Post 29 Oct 2015, 11:01
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland
I have created the PE formatter in form of macros for fasm g, and so I'm almost ready to make fasm g self-hosting on Windows platform. The macros are mostly compatible with syntax of fasm 1, except for the "format PE" line with settings, but I may it that later.

I'm attaching current version of macros together with adapted "PE demo" that comes with fasm 1. You may notice that even the fixups can be created this way - if the MZ examples were not enough, this definitely demonstrates that it is possible to create code with relocations using just macros in fasm g.

There is also no checksum computation in this version, but this should be very easy to add.


Description: PE demo from fasm package, adapted to fasm g
Download
Filename: pedemo.zip
Filesize: 10.64 KB
Downloaded: 52 Time(s)

Post 30 Oct 2015, 18:42
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland
The PE macros and Win32 example are now part of the official fasm g package.

I have also prepared the complete set of macros that allow fasm g to assemble itself in its Win32 version. I still need to decide how to organize the files to include the self-hosting option in the official package, but for now I attach the required files here for anyone interested. The "selfhost" folder should be placed inside "source/win32".


Description: Macros for fasm g self-assembly
Download
Filename: selfhost.zip
Filesize: 13 KB
Downloaded: 50 Time(s)

Post 31 Oct 2015, 22:12
View user's profile Send private message Visit poster's website Reply with quote
idle



Joined: 06 Jan 2011
Posts: 340
Location: ukraina
That's is strong, you are cool!
Post 01 Nov 2015, 12:42
View user's profile Send private message Send e-mail Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland
The latest fasm g package includes the self-hosting under Win32. The altered source exploits the quirks of fasm syntax to detect whether it is being assembled by fasm 1 or fasm g and include appropriate headers. The tricky part goes like this:

Code:
match ,{

        ; fasm 1 detected
        include 'win32a.inc'

match -,{
else

        ; fasm g detected
        include 'selfhost.inc'

end match
_ equ }

Next in the queue are ELF executable format macros and self-hosting Linux version. This one should be easier.
Post 02 Nov 2015, 15:54
View user's profile Send private message Visit poster's website Reply with quote
idle



Joined: 06 Jan 2011
Posts: 340
Location: ukraina
Tomasz, fasmg expected to be more flexible - do you plan to add include once, multiline comments, command line access, fasmg ide, %time.YYYY/MM/DD assembly var?
Post 03 Nov 2015, 09:13
View user's profile Send private message Send e-mail Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6356
Location: Kraków, Poland
fasm g is a bare engine that still is enough to implement everything in form of macros, and can be therefore seen as an ultimate embodiment of my "complex solutions with simple features" idea. All the added features are the ones that make it possible to do even more things with macros only. The implementation of multi-line comments with a very simple macro is shown in the manual, various variants of "include once" (depending on what problem you look to solve) are also easy to make. I planned to add more built-in parameters but I do not want to clutter the namespace too much so I decided to wait until I see which ones I really need. I do not have plans for any new IDE and I do not know what you meant by "command line access".

What I wanted to achieve with fasm g is perhaps a demonstration of my own concept of what an assembler is (the idea I originally came up with in my "Understanding fasm" text.
Post 03 Nov 2015, 10:54
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >

Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2005 phpBB Group.

Main index   Download   Documentation   Examples   Message board
Copyright © 2004-2016, Tomasz Grysztar.