flat assembler
Message board for the users of flat assembler.
 Home   FAQ   Search   Register 
 Profile   Log in to check your private messages   Log in 
flat assembler > Windows > [Solved] Linker issues while embedding FASM in C++ library

Author
Thread Post new topic Reply to topic
A$M



Joined: 29 Feb 2012
Posts: 94
[Solved] Linker issues while embedding FASM in C++ library
Hi there!

I've been working on a small library which allows you to use FASM to assemble code at runtime from C++. Surprisingly, it works almost flawlessly! But sadly, there is an issue on x64. Since FASM was made in x86, it needs to be converted to x64. Thankfully, the 64-bit Linux version has a file named "modes.inc" which solves most incompatibilities, so the code assembles fine. I'm using MS64 COFF (.obj files) as the output format so I can easily link with C++ code on Visual Studio 2017.

However, I get the linker error LNK2017. MSDN says:

Quote:
'symbol' relocation to 'segment' invalid without /LARGEADDRESSAWARE:NO

You are trying to build a 64-bit image with 32-bit addresses. To do this, you must:

- Use a fixed load address.
- Restrict the image to 3 GB.
- Specify /largeaddressaware:no.


Well, that makes sense. After some research, I found that is because of code like this:

Code:
mov eaxsome_label


Which should be rewritten as:

Code:
lea eax, [some_label]


And FASM is full of code like that. It happens probably 136 times. At least that's the amount of ADDR32 relocations reported by dumpbin. So it'd take a long time to track all of them down and fix them.

I thought that would be easy to solve with macros. Maybe something like this:

Code:
macro mov destsrc
{
    if src eqtype mylabel
        lea dest, [src]
    else
        mov destsrc
    end if
}
mylabel:


But it turns out that eqtype can't differentiate between literals and labels, so I'm clueless if such macro would even be possible.

So what about the MSDN's fixes?

1. To be able to use the /FIXED flag I had to link the .obj with the executable instead of the library and it didn't work. Even if it had worked I suspect it would cause more problems than it would solve.
2. How do I even do that? I think that has something to do with PE32+, but I'm not sure.
3. That works and is how I've been doing it until now. But it limits memory to 2 GB for the whole executable which is unacceptable for a 64-bit program.

So I'd like your advice.
Can I use a macro to fix these issues? How?
Could I use another format instead of MS64 COFF?
Maybe have a second executable running in parallel which would do the assembling?
Do I need to fix the code manually?

I have no idea.

Thanks for your time.


Last edited by A$M on 04 May 2017, 13:48; edited 1 time in total
Post 01 May 2017, 23:46
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 876
A$M
As long as you compile that to an obj file, the labels are relocatable and hence do not have absolute values (e.g., you cannot divide such a label by a constant). Therefore relativeto is your friend here. First make sure the argument is of an expression type via eqtype, then make sure that expression has a non-absolute value by comparing it with relativeto 0. This will not catch expressions with registers though, so you also may choose to compare relativeto some_label_with_same_base_as_needed, but this wouldn't be necessary for mov. Things may get different with push, if the code you compile employs its dirty features. Smile

_________________
Faith is a superposition of knowledge and fallacy
Post 02 May 2017, 20:48
View user's profile Send private message Reply with quote
A$M



Joined: 29 Feb 2012
Posts: 94

l_inc wrote:
A$M
As long as you compile that to an obj file, the labels are relocatable and hence do not have absolute values (e.g., you cannot divide such a label by a constant). Therefore relativeto is your friend here. First make sure the argument is of an expression type via eqtype, then make sure that expression has a non-absolute value by comparing it with relativeto 0. This will not catch expressions with registers though, so you also may choose to compare relativeto some_label_with_same_base_as_needed, but this wouldn't be necessary for mov. Things may get different with push, if the code you compile employs its dirty features. Smile

Thank you very much! That's exactly what I needed. I was going to use DLLs which would be a pain in the ass.

So this is what I did. First, a "purge mov" because "modes.inc" already defines it. Then this:

Code:
macro mov destsrc
{
        if src eq esp
                mov destESP    ; This is from modes.inc. I'm not sure what it does but I'm keeping it
        else if src eqtype 0 & ~src relativeto 0
                lea r8d, [src]
                mov destr8d
        else
                mov destsrc
        end if
}

However, that causes an error: relative jump out of range. I guess that's because my macro increased the distance between the loop instruction and the label, so I did this:

Code:
macro loop dest
{
        dec ecx
        jnz dest
}

And it assembles again. Now let's see how many errors I have... 59 errors. Well, that's much better. Of course, mov is not the only instruction causing problems, so I'm going to have to make guesses. So far I found mov, cmp, add & sub. There are still many, but I'm slowly getting there.

Thank you again, l_inc.
Post 02 May 2017, 23:32
View user's profile Send private message Reply with quote
A$M



Joined: 29 Feb 2012
Posts: 94
Ok, so I managed to make all the macros and it linked properly. However, I didn't know that everything was loaded in 32-bit inaccessible memory (RIP for main is 0x00007FF6985589C0) so everything crashes and explodes. For example, "lea eax, variable / mov dword[eax], 123" fails for obvious reasons.

How could I fix this? Maybe if I use more macros to make everything 64-bit. But I think I'm either giving up or at least shelving this project for now. Good thing I've been doing this only since Saturday because it's a simple port so there's not much harm.

But still, thanks again for your help. :-D
Post 03 May 2017, 03:12
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 876
A$M
My ability to think is very limited at the afterwork hours, but it seems you are pointing out a bug in fasm:

Code:
use64
org $100000000
mov eax,$$
lea eax,[$$]


None of the above two instructions should compile, but the overflow is caught for mov only.

_________________
Faith is a superposition of knowledge and fallacy
Post 03 May 2017, 20:33
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 876
OK. I thought a few more times about that, and this is probably not a bug, as fasm has never been conservative enough to expect an explicit "cast" with things like lea ax,[eax]. In fact, such a cast already has a different effect assigned to it, which is forcing a longer immediate encoding.

The problem with the above code is that rip in the address expression became implicit similarly to how ip has always been implicit in control transfer instructions with ip-relative addressing. So I see a bit of an inconsistency in that the overflow check is performed in the following code, but not in the code above.

Code:
org $10000
jmp $


_________________
Faith is a superposition of knowledge and fallacy
Post 03 May 2017, 21:30
View user's profile Send private message Reply with quote
A$M



Joined: 29 Feb 2012
Posts: 94

l_inc wrote:
A$M
My ability to think is very limited at the afterwork hours, but it seems you are pointing out a bug in fasm:

Code:
use64
org $100000000
mov eax,$$
lea eax,[$$]


None of the above two instructions should compile, but the overflow is caught for mov only.

Is that so? Well, if FASM had given me an error for stuff like "lea eax, [label]", that would've saved me some frustration. Oh well.

I said I would be shelving this or giving up, but I changed my mind. I have a new hope. I'm now doing something like this:

Code:

use64
org 0x10000

        dq asm_test    ; This is how I get the public symbols from C++
        dq val1
        dq val2

asm_test:
        add rsp8*5

        mov raxval1
        mov qword[rax], 25

        lea eax, [val2]
        mov qword[eax], 50

        sub rsp8*5
        ret

        val1 dq 0
        val2 dq 0

Then, from C++ I just allocate executable memory at 0x10000, load this from a file, copy there and run. Works flawlessly. This, of course, is just a little test. But if it works for this, it should work for my project.

Thanks again for your help. I'll update shortly.
Post 03 May 2017, 21:36
View user's profile Send private message Reply with quote
A$M



Joined: 29 Feb 2012
Posts: 94

l_inc wrote:
OK. I thought a few more times about that, and this is probably not a bug, as fasm has never been conservative enough to expect an explicit "cast" with things like lea ax,[eax]. In fact, such a cast already has a different effect assigned to it, which is forcing a longer immediate encoding.

The problem with the above code is that rip in the address expression became implicit similarly to how ip has always been implicit in control transfer instructions with ip-relative addressing. So I see a bit of an inconsistency in that the overflow check is performed in the following code, but not in the code above.

Code:
org $10000
jmp $


Oh, ok, I see. Everything is working fine now. Thanks for your help.
Post 04 May 2017, 13:47
View user's profile Send private message Reply with quote
A$M



Joined: 29 Feb 2012
Posts: 94
IDK if this is against any rules, but I'm using this post to say I've published this library on GitHub. I've called it fasmcpp.
Post 10 May 2017, 01:03
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >

Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2005 phpBB Group.

Main index   Download   Documentation   Examples   Message board
Copyright © 2004-2016, Tomasz Grysztar.