flat assembler
Message board for the users of flat assembler.

Index > Windows > [Solved] Linker issues while embedding FASM in C++ library

Author
Thread Post new topic Reply to topic
A$M



Joined: 29 Feb 2012
Posts: 94
A$M 01 May 2017, 23:46
Hi there!

I've been working on a small library which allows you to use FASM to assemble code at runtime from C++. Surprisingly, it works almost flawlessly! But sadly, there is an issue on x64. Since FASM was made in x86, it needs to be converted to x64. Thankfully, the 64-bit Linux version has a file named "modes.inc" which solves most incompatibilities, so the code assembles fine. I'm using MS64 COFF (.obj files) as the output format so I can easily link with C++ code on Visual Studio 2017.

However, I get the linker error LNK2017. MSDN says:
Quote:
'symbol' relocation to 'segment' invalid without /LARGEADDRESSAWARE:NO

You are trying to build a 64-bit image with 32-bit addresses. To do this, you must:

- Use a fixed load address.
- Restrict the image to 3 GB.
- Specify /largeaddressaware:no.

Well, that makes sense. After some research, I found that is because of code like this:
Code:
mov eax, some_label    

Which should be rewritten as:
Code:
lea eax, [some_label]    

And FASM is full of code like that. It happens probably 136 times. At least that's the amount of ADDR32 relocations reported by dumpbin. So it'd take a long time to track all of them down and fix them.

I thought that would be easy to solve with macros. Maybe something like this:
Code:
macro mov dest, src
{
    if src eqtype mylabel
        lea dest, [src]
    else
        mov dest, src
    end if
}
mylabel:    

But it turns out that eqtype can't differentiate between literals and labels, so I'm clueless if such macro would even be possible.

So what about the MSDN's fixes?

1. To be able to use the /FIXED flag I had to link the .obj with the executable instead of the library and it didn't work. Even if it had worked I suspect it would cause more problems than it would solve.
2. How do I even do that? I think that has something to do with PE32+, but I'm not sure.
3. That works and is how I've been doing it until now. But it limits memory to 2 GB for the whole executable which is unacceptable for a 64-bit program.

So I'd like your advice.
Can I use a macro to fix these issues? How?
Could I use another format instead of MS64 COFF?
Maybe have a second executable running in parallel which would do the assembling?
Do I need to fix the code manually?

I have no idea.

Thanks for your time.


Last edited by A$M on 04 May 2017, 13:48; edited 1 time in total
Post 01 May 2017, 23:46
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 02 May 2017, 20:48
A$M
As long as you compile that to an obj file, the labels are relocatable and hence do not have absolute values (e.g., you cannot divide such a label by a constant). Therefore relativeto is your friend here. First make sure the argument is of an expression type via eqtype, then make sure that expression has a non-absolute value by comparing it with relativeto 0. This will not catch expressions with registers though, so you also may choose to compare relativeto some_label_with_same_base_as_needed, but this wouldn't be necessary for mov. Things may get different with push, if the code you compile employs its dirty features. Smile

_________________
Faith is a superposition of knowledge and fallacy
Post 02 May 2017, 20:48
View user's profile Send private message Reply with quote
A$M



Joined: 29 Feb 2012
Posts: 94
A$M 02 May 2017, 23:32
l_inc wrote:
A$M
As long as you compile that to an obj file, the labels are relocatable and hence do not have absolute values (e.g., you cannot divide such a label by a constant). Therefore relativeto is your friend here. First make sure the argument is of an expression type via eqtype, then make sure that expression has a non-absolute value by comparing it with relativeto 0. This will not catch expressions with registers though, so you also may choose to compare relativeto some_label_with_same_base_as_needed, but this wouldn't be necessary for mov. Things may get different with push, if the code you compile employs its dirty features. Smile
Thank you very much! That's exactly what I needed. I was going to use DLLs which would be a pain in the ass.

So this is what I did. First, a "purge mov" because "modes.inc" already defines it. Then this:
Code:
macro mov dest, src
{
        if src eq esp
                mov dest, ESP    ; This is from modes.inc. I'm not sure what it does but I'm keeping it
        else if src eqtype 0 & ~src relativeto 0
                lea r8d, [src]
                mov dest, r8d
        else
                mov dest, src
        end if
}    
However, that causes an error: relative jump out of range. I guess that's because my macro increased the distance between the loop instruction and the label, so I did this:
Code:
macro loop dest
{
        dec ecx
        jnz dest
}    
And it assembles again. Now let's see how many errors I have... 59 errors. Well, that's much better. Of course, mov is not the only instruction causing problems, so I'm going to have to make guesses. So far I found mov, cmp, add & sub. There are still many, but I'm slowly getting there.

Thank you again, l_inc.
Post 02 May 2017, 23:32
View user's profile Send private message Reply with quote
A$M



Joined: 29 Feb 2012
Posts: 94
A$M 03 May 2017, 03:12
Ok, so I managed to make all the macros and it linked properly. However, I didn't know that everything was loaded in 32-bit inaccessible memory (RIP for main is 0x00007FF6985589C0) so everything crashes and explodes. For example, "lea eax, variable / mov dword[eax], 123" fails for obvious reasons.

How could I fix this? Maybe if I use more macros to make everything 64-bit. But I think I'm either giving up or at least shelving this project for now. Good thing I've been doing this only since Saturday because it's a simple port so there's not much harm.

But still, thanks again for your help. :-D
Post 03 May 2017, 03:12
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 03 May 2017, 20:33
A$M
My ability to think is very limited at the afterwork hours, but it seems you are pointing out a bug in fasm:
Code:
use64
org $100000000
mov eax,$$
lea eax,[$$]    

None of the above two instructions should compile, but the overflow is caught for mov only.

_________________
Faith is a superposition of knowledge and fallacy
Post 03 May 2017, 20:33
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc 03 May 2017, 21:30
OK. I thought a few more times about that, and this is probably not a bug, as fasm has never been conservative enough to expect an explicit "cast" with things like lea ax,[eax]. In fact, such a cast already has a different effect assigned to it, which is forcing a longer immediate encoding.

The problem with the above code is that rip in the address expression became implicit similarly to how ip has always been implicit in control transfer instructions with ip-relative addressing. So I see a bit of an inconsistency in that the overflow check is performed in the following code, but not in the code above.
Code:
org $10000
jmp $    

_________________
Faith is a superposition of knowledge and fallacy
Post 03 May 2017, 21:30
View user's profile Send private message Reply with quote
A$M



Joined: 29 Feb 2012
Posts: 94
A$M 03 May 2017, 21:36
l_inc wrote:
A$M
My ability to think is very limited at the afterwork hours, but it seems you are pointing out a bug in fasm:
Code:
use64
org $100000000
mov eax,$$
lea eax,[$$]    

None of the above two instructions should compile, but the overflow is caught for mov only.
Is that so? Well, if FASM had given me an error for stuff like "lea eax, [label]", that would've saved me some frustration. Oh well.

I said I would be shelving this or giving up, but I changed my mind. I have a new hope. I'm now doing something like this:
Code:
use64
org 0x10000

        dq asm_test    ; This is how I get the public symbols from C++
        dq val1
        dq val2

asm_test:
        add rsp, 8*5

        mov rax, val1
        mov qword[rax], 25

        lea eax, [val2]
        mov qword[eax], 50

        sub rsp, 8*5
        ret

        val1 dq 0
        val2 dq 0    
Then, from C++ I just allocate executable memory at 0x10000, load this from a file, copy there and run. Works flawlessly. This, of course, is just a little test. But if it works for this, it should work for my project.

Thanks again for your help. I'll update shortly.
Post 03 May 2017, 21:36
View user's profile Send private message Reply with quote
A$M



Joined: 29 Feb 2012
Posts: 94
A$M 04 May 2017, 13:47
l_inc wrote:
OK. I thought a few more times about that, and this is probably not a bug, as fasm has never been conservative enough to expect an explicit "cast" with things like lea ax,[eax]. In fact, such a cast already has a different effect assigned to it, which is forcing a longer immediate encoding.

The problem with the above code is that rip in the address expression became implicit similarly to how ip has always been implicit in control transfer instructions with ip-relative addressing. So I see a bit of an inconsistency in that the overflow check is performed in the following code, but not in the code above.
Code:
org $10000
jmp $    
Oh, ok, I see. Everything is working fine now. Thanks for your help.
Post 04 May 2017, 13:47
View user's profile Send private message Reply with quote
A$M



Joined: 29 Feb 2012
Posts: 94
A$M 10 May 2017, 01:03
IDK if this is against any rules, but I'm using this post to say I've published this library on GitHub. I've called it fasmcpp.
Post 10 May 2017, 01:03
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.