flat assembler
Message board for the users of flat assembler.
 Home   FAQ   Search   Register 
 Profile   Log in to check your private messages   Log in 
flat assembler > Compiler Internals > fasmg - bugs in x86-64 macros

Author
Thread Post new topic Reply to topic
tthsqe



Joined: 20 May 2009
Posts: 701
fasmg - bugs in x86-64 macros
for completeness put a

Code:
include 'x86/include/format/format.inc'
include 'x86/include/avx.inc'

format ELF64 executable
entry Start

segment executable readable
Start:


at the beginning of all of these examples. I also had to change some of the recursive includes go get fasmg working out of the box.

here are some cases where fasmg does not match fasm1

Not smallest encoding

Code:
mov rax0xFFFFFFFFFF000000


If I sign extended the imm by subtracting 2^64, then it gives smallest and matches fasm1.

internal error

Code:
mov r8ddword[rbp+10]  ; ok
mov r8ddword[rbp+100; ok
mov r8ddword[rbp+1000]; Error: symbol 'index' is undefined or out of scope.



However, on this last example, i have some some macros that do "mov r8d, dword[rbp+foo.bar]". This does compile but it compiles to "mov r8d, dword[rip+foo.bar]". This makes me believe it is a initialisation error in index.

I will keep posting more bugs as I find them.
Post 15 Aug 2017, 21:00
View user's profile Send private message Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 701
same problem with lea. The "<" is fasmg and the ">" is fasm1:

Code:

<    15081:     48 8d 35 28 ff ff ff    lea    rsi,[rip+0xffffffffffffff28]
>    15081:     48 8d b5 28 ff ff ff    lea    rsi,[rbp-0xd8]


Post 15 Aug 2017, 21:52
View user's profile Send private message Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 701
Out of a 100kb exe, besides the mov rax, ... issue, this are the only differences between fasmg (<) and fasm1 (>). Obviously the output from fasmg is a complete error in these cases. They all use rbp as the base of an address.

Code:
<     baa5:     44 8b 05 d0 01 00 00    mov    r8d,DWORD PTR [rip+0x1d0]        # 0xbc7c
---
>     baa5:     44 8b 85 d0 01 00 00    mov    r8d,DWORD PTR [rbp+0x1d0]
14999c14999
<     bab0:     4c 03 05 00 02 00 00    add    r8,QWORD PTR [rip+0x200]        # 0xbcb7
---
>     bab0:     4c 03 85 00 02 00 00    add    r8,QWORD PTR [rbp+0x200]
15702c15702
<     c394:     89 35 d0 01 00 00       mov    DWORD PTR [rip+0x1d0],esi        # 0xc56a
---
>     c394:     89 b5 d0 01 00 00       mov    DWORD PTR [rbp+0x1d0],esi
18579c18579
<     eca9:     48 03 05 f8 01 00 00    add    rax,QWORD PTR [rip+0x1f8]        # 0xeea8
---
>     eca9:     48 03 85 f8 01 00 00    add    rax,QWORD PTR [rbp+0x1f8]
19132c19132
<     f542:     4c 8b 05 08 02 00 00    mov    r8,QWORD PTR [rip+0x208]        # 0xf751
---
>     f542:     4c 8b 85 08 02 00 00    mov    r8,QWORD PTR [rbp+0x208]
19139c19139
<     f566:     44 8b 05 d0 01 00 00    mov    r8d,DWORD PTR [rip+0x1d0]        # 0xf73d
---
>     f566:     44 8b 85 d0 01 00 00    mov    r8d,DWORD PTR [rbp+0x1d0]
19141c19141
<     f571:     4c 03 05 00 02 00 00    add    r8,QWORD PTR [rip+0x200]        # 0xf778
---
>     f571:     4c 03 85 00 02 00 00    add    r8,QWORD PTR [rbp+0x200]
21190c21190
<    1131b:     4c 8b 05 08 02 00 00    mov    r8,QWORD PTR [rip+0x208]        # 0x1152a
---
>    1131b:     4c 8b 85 08 02 00 00    mov    r8,QWORD PTR [rbp+0x208]
21197c21197
<    1133f:     44 8b 05 d0 01 00 00    mov    r8d,DWORD PTR [rip+0x1d0]        # 0x11516
---
>    1133f:     44 8b 85 d0 01 00 00    mov    r8d,DWORD PTR [rbp+0x1d0]
21199c21199
<    1134a:     4c 03 05 00 02 00 00    add    r8,QWORD PTR [rip+0x200]        # 0x11551
---
>    1134a:     4c 03 85 00 02 00 00    add    r8,QWORD PTR [rbp+0x200]
22280c22280
<    1239c:     4c 8b 05 08 02 00 00    mov    r8,QWORD PTR [rip+0x208]        # 0x125ab
---
>    1239c:     4c 8b 85 08 02 00 00    mov    r8,QWORD PTR [rbp+0x208]
22287c22287
<    123c0:     44 8b 05 d0 01 00 00    mov    r8d,DWORD PTR [rip+0x1d0]        # 0x12597
---
>    123c0:     44 8b 85 d0 01 00 00    mov    r8d,DWORD PTR [rbp+0x1d0]
22289c22289
<    123cb:     4c 03 05 00 02 00 00    add    r8,QWORD PTR [rip+0x200]        # 0x125d2
---
>    123cb:     4c 03 85 00 02 00 00    add    r8,QWORD PTR [rbp+0x200]
24993c24993
<    14a25:     48 8b 1d e0 01 00 00    mov    rbx,QWORD PTR [rip+0x1e0]        # 0x14c0c
---
>    14a25:     48 8b 9d e0 01 00 00    mov    rbx,QWORD PTR [rbp+0x1e0]
25114c25114
<    14bf8:     0f b6 05 d0 01 00 00    movzx  eax,BYTE PTR [rip+0x1d0]        # 0x14dcf
---
>    14bf8:     0f b6 85 d0 01 00 00    movzx  eax,BYTE PTR [rbp+0x1d0]
25168c25168
<    14cab:     44 89 25 dc 01 00 00    mov    DWORD PTR [rip+0x1dc],r12d        # 0x14e8e
---
>    14cab:     44 89 a5 dc 01 00 00    mov    DWORD PTR [rbp+0x1dc],r12d
25246c25246
<    14da7:     89 0d d0 01 00 00       mov    DWORD PTR [rip+0x1d0],ecx        # 0x14f7d
---
>    14da7:     89 8d d0 01 00 00       mov    DWORD PTR [rbp+0x1d0],ecx
25461c25461
<    15081:     48 8d 35 28 ff ff ff    lea    rsi,[rip+0xffffffffffffff28]        # 0x14fb0
---
>    15081:     48 8d b5 28 ff ff ff    lea    rsi,[rbp-0xd8]
25468,25469c25468,25469
<    150a1:     48 8b 35 e8 01 00 00    mov    rsi,QWORD PTR [rip+0x1e8]        # 0x15290
<    150a8:     4c 8b 35 e0 01 00 00    mov    r14,QWORD PTR [rip+0x1e0]        # 0x1528f
---
>    150a1:     48 8b b5 e8 01 00 00    mov    rsi,QWORD PTR [rbp+0x1e8]
>    150a8:     4c 8b b5 e0 01 00 00    mov    r14,QWORD PTR [rbp+0x1e0]

Post 15 Aug 2017, 23:00
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6676
Location: Kraków, Poland
I made two little corrections in x64.inc, please try it now.
Post 16 Aug 2017, 08:24
View user's profile Send private message Visit poster's website Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 701
Identical binaries! BTW, fasm1 assembles the source in 0.1 seconds. fasmg used to incorrectly assemble the source in 18 seconds. Now fasmg correctly assembles the source in 21 seconds.
I'm not sure how to get this time down.

Do you know if anyone has started writing macros for arm-v8 (64 bit arm)?

Also, could you put the mach-o formatters in the main distribution?
Post 16 Aug 2017, 11:44
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6676
Location: Kraków, Poland

tthsqe wrote:
BTW, fasm1 assembles the source in 0.1 seconds. fasmg used to incorrectly assemble the source in 18 seconds. Now fasmg correctly assembles the source in 21 seconds.
I'm not sure how to get this time down.

I have been warning about this is since the beginning - implementing a complex instruction sets like x86 entirely in form of macros is going to be slow no matter what. You could, however, try to optimize some of them, perhaps making some compromises in their feature set, to get the assembly time a little bit down. Reducing a number of lines in any macro should have a positive effect on performance, especially in the macros that get called extensively. For example this one from x64.inc:

Code:
macro x86.parse_operand ns,op
        ns.size = 0
        match prefix valueop
                match :szx86.prefix
                        ns.size = sz
                        x86.parse_operand_value ns,value
                else
                        x86.parse_operand_value ns,op
                end match
        else
                x86.parse_operand_value ns,op
        end match
end macro

can be made simpler:

Code:
macro x86.parse_operand ns,op
        ns.size = 0
        match :sz valuex86.op
                ns.size = sz
                x86.parse_operand_value ns,value
        else
                x86.parse_operand_value ns,op
        end match
end macro

and even this reduces the assembly speed a little bit (but it breaks some of the tricks with equates).


tthsqe wrote:
Do you know if anyone has started writing macros for arm-v8 (64 bit arm)?

I was hoping that revolution could try, having so much experience from making FASMARM, but it seems they have not enough spare time to play with fasmg. Honestly, I would prefer to leave this to people more competent in ARM architecture than me, but there have been no volunteers so far.


tthsqe wrote:
Also, could you put the mach-o formatters in the main distribution?

I would like to have them tester and confirmed a bit more before putting them out for general use. But I may include the self-hosted MacOS version of fasmg (which includes these macros) in the main package.
Post 16 Aug 2017, 13:47
View user's profile Send private message Visit poster's website Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 701
I will try to get a simple hello world working for arm64. As far I can see, this involves getting a working subset of the instruction set as well as modifying your elfexe macros
Post 16 Aug 2017, 15:14
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6676
Location: Kraków, Poland

tthsqe wrote:
I will try to get a simple hello world working for arm64. As far I can see, this involves getting a working subset of the instruction set as well as modifying your elfexe macros

The ELF.Settings namespace allows to set up some of the values in headers without modifying the ELFEXE.INC. If you'd need some additional modifications, it would be good to make them also in form of settings.
Post 16 Aug 2017, 15:42
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 15301
Location: Bigweld Industries

Tomasz Grysztar wrote:

tthsqe wrote:
Do you know if anyone has started writing macros for arm-v8 (64 bit arm)?

I was hoping that revolution could try, having so much experience from making FASMARM, but it seems they have not enough spare time to play with fasmg. Honestly, I would prefer to leave this to people more competent in ARM architecture than me, but there have been no volunteers so far.

It is a lot of work. I briefly looked into it and discovered that. For the basic 32-bit V1 it might be not too bad in terms of time required, but it quickly gets complicated after that. FWIW I can't yet justify the amplified assembly times compared to fasm1. But if someone wants to give it a go they can use the verifiers I include with fasmarm to check that they are on the right track. That might speed things up a little.
Post 16 Aug 2017, 19:35
View user's profile Send private message Visit poster's website Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 701
Thomasz, how well is arm syntax going to play with fasmg?
For example, this is valid aarch64 syntax

Code:
ldr x0, [x1, :lo12:a+3]
stp x0x1, [sp, -16]!

Post 17 Aug 2017, 11:38
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6676
Location: Kraków, Poland
You can MATCH any combination of special characters and other symbol with no problem, except for the semicolon and backslash at the end of line, as they are processed and stripped by the line reader. Also the "#" has a special meaning in fasmg's identifier syntax, but you can MATCH it away before it goes to the expression evaluator.

My introductory manual for fasmg contains several examples of processing various kind of syntax (though they focus on Z80 architecture, as it was especially fitting to showcase some of the methods). I may add more in the future. In fact, if you encounter any trouble applying these tricks to ARM syntax, please let me know and I may expand the documentation with explained solutions.

Now as I think of it, it might be a good idea to add a section about processing values that contain both relocatable addresses and registers, as it happens when x86 instructions are combined with object output formats.
Post 17 Aug 2017, 12:10
View user's profile Send private message Visit poster's website Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 701
Hello world is working at https://github.com/tthsqe12/asm/blob/master/armfish.arm
Thomasz, if you have time, could you make sure I'm on the right track with the macros? There are in arm/include/aarch64.inc
EDIT: I simply cannot find good enough information on relocations, so I will be completely ignoring them and only concentrating on the elfexe format.
Post 17 Aug 2017, 21:37
View user's profile Send private message Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 701
There is another inconsistency between fasm1 and fasmg. With

Code:
format PE64 console


I had to apply the following patch to get fasmg to match fasm1.

Code:
else match =PE64? settingsclause:
                PE.Settings.Magic = 0x20B
                PE.Settings.Machine = IMAGE_FILE_MACHINE_AMD64
 -              PE.Settings.Characteristics = IMAGE_FILE_EXECUTABLE_IMAGE or IMAGE_FILE_32BIT_MACHINE or IMAGE_FILE_LINE_NUMS_STRIPPED or IMAGE_FILE_LOCAL_SYMS_STRIPPED
 +              PE.Settings.Characteristics = IMAGE_FILE_EXECUTABLE_IMAGE or IMAGE_FILE_LARGE_ADDRESS_AWARE or IMAGE_FILE_LINE_NUMS_STRIPPED or IMAGE_FILE_LOCAL_SYMS_STRIPPED
                PE.Settings.DllCharacteristics = 0
                PE.Settings.MajorSubsystemVersion = 5
                PE.Settings.MinorSubsystemVersion = 0


I think fasm1 is right here.
Post 04 Oct 2017, 07:21
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6676
Location: Kraków, Poland

tthsqe wrote:
There is another inconsistency between fasm1 and fasmg. (...)

Thanks. I'm correcting it both in fasmg's basic examples and in the Windows headers package. They now have different but compatible implementations of "format" macro.
Post 04 Oct 2017, 09:04
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >

Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2005 phpBB Group.

Main index   Download   Documentation   Examples   Message board
Copyright © 2004-2016, Tomasz Grysztar.