Creating Console DLL 64-bit

Index > Windows > Creating Console DLL 64-bit

Author

Thread

system error

Joined: 01 Sep 2013
Posts: 670

system error 12 Mar 2015, 13:20

How to properly create a console-based PE64 DLL? I managed to make one for 32-bit but not in 64-bit Windows 8. I am using FASMW 1.71.34

My code structure looks something like this

Code:

;File: thelib64.asm
format PE64 console DLL
entry DllEntryPoint
include 'win64axp.inc'

section '.text' code readable executable

DllEntryPoint:
        sub rsp,8
        mov eax,1
        ret 24 ;?

;The other stuff

section '.edata' export data readable
        export 'thelib64.dll',\
       ;The other stuff names and aliases

section '.reloc' fixups data readable discardable ;switch this ON and OFF but still fail
;empty

section '.idata' import data readable
;some import stuff from msvcrt.dll

I call it from this code;

Code:

;File: calldll.asm
format PE64 console
include 'win64axp.inc'
entry start

section '.text' code readable executable
start:
      
      mov rcx,[x]
      call [theOtherStuff]  
      call [exit]

section '.data' data readable writeable
x dq 34

section '.idata' import data readable
  library thelib64,'thelib64.dll'
  import thelib64,\
         ;the Other stuff import

The errors are either

1. ..."Not started correctly..."
2. ..."Not a win64 application... contact vendor bla bla..."

My questions;

1. What's wrong? The same code structure works perfectly in 32-bit version

2. Is it even possible to create a console program by calling win64 header like this, considering the same code structure working just fine in 32-bit version.

12 Mar 2015, 13:20

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system

revolution 12 Mar 2015, 13:29

I'm assuming you already used a search engine and came up with no answers. So based upon that assumption and your code above I'd suggest that a 64-bit console DLL is not supported.

Therefore, if you still need to have a console window for your DLL then you can use AllocConsole. But the new console won't be attached to the local console window.

12 Mar 2015, 13:29

system error

Joined: 01 Sep 2013
Posts: 670

system error 12 Mar 2015, 13:34

revolution wrote:

I'm assuming you already used a search engine and came up with no answers. So based upon that assumption and your code above I'd suggest that a 64-bit console DLL is not supported.

Therefore, if you still need to have a console window for your DLL then you can use AllocConsole. But the new console won't be attached to the local console window.

Most examples on the web shows GUI version. Too bad.

But the same code works for 32-bit version on the same windows 8 (64-bit). That's weird though.

Is there any code 'fix' for this case?

12 Mar 2015, 13:34

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system

revolution 12 Mar 2015, 13:41

If you don't need a separate console window for your DLL then just make the DLL a GUI. You can still link to GUI DLLs from a console application, everything will still work.

12 Mar 2015, 13:41

l_inc

Joined: 23 Oct 2009
Posts: 881

l_inc 12 Mar 2015, 13:42

revolution

Quote:

I'm assuming you already used a search engine

A very rarely justified assumption. He did not. It's the well known empty section problem.

_________________
Faith is a superposition of knowledge and fallacy

12 Mar 2015, 13:42

system error

Joined: 01 Sep 2013
Posts: 670

system error 12 Mar 2015, 16:58

l_inc wrote:

It's the well known empty section problem.

...with very little reference sadly. Every web findings didn't really say it's a 'problem'. Some offer workaround to make the 'problem' disappear. But they all don't work in this particular case. That's why this thread. LOL.

12 Mar 2015, 16:58

system error

Joined: 01 Sep 2013
Posts: 670

system error 12 Mar 2015, 17:24

revolution wrote:

If you don't need a separate console window for your DLL then just make the DLL a GUI. You can still link to GUI DLLs from a console application, everything will still work.

Didn't do the trick. If it works for 32-bit version on the same console, I don't see why 64-bit version shouldn't work. They are both on the same Win8 anyway. I am thinking that maybe I am missing some basic steps in creating 64-bit dll. I don't know.

12 Mar 2015, 17:24

l_inc

Joined: 23 Oct 2009
Posts: 881

l_inc 12 Mar 2015, 18:01

system error

Quote:

...with very little reference sadly

Look here, here, here and here. I hope, that's enough reference.

_________________
Faith is a superposition of knowledge and fallacy

12 Mar 2015, 18:01

system error

Joined: 01 Sep 2013
Posts: 670

system error 12 Mar 2015, 18:12

l_inc wrote:

system error

Quote:
...with very little reference sadly

Look here, here, here and here. I hope, that's enough reference.

LOL. Where do you think I learned creating my 32-bit DLL from?
But some of these don't work quite well in my 64-bit library.

One more question;

Since I don't have the program with me right now, I can't test whether I should use ExitProcess or just exit from msvcrt? It works fine in the 32-bit version. But I am not very sure anymore for the 64-bit version because this is a kind of cross-call between win64 and console. Could that be one of the problem?

12 Mar 2015, 18:12

l_inc

Joined: 23 Oct 2009
Posts: 881

l_inc 12 Mar 2015, 18:18

system error

Quote:

LOL. Where do you think I learned creating my 32-bit DLL from?

No idea, but if you followed at least one suggestion to make your relocations section not empty, you wouldn't have the problem.

Quote:

this is a kind of cross-call between win64 and console

Oh, come on! GUI or console in a dll makes no difference. You can put any with no consequences.

_________________
Faith is a superposition of knowledge and fallacy

12 Mar 2015, 18:18

system error

Joined: 01 Sep 2013
Posts: 670

system error 12 Mar 2015, 18:29

l_inc wrote:

No idea, but if you followed at least one suggestion to make your relocations section not empty, you wouldn't have the problem.

I did in 32-bit. Works.
I did in 64-bit. FAIL.

l_inc wrote:

Oh, come on! GUI or console in a dll makes no difference. You can put any with no consequences.

Really? Ok then Very Happy

12 Mar 2015, 18:29

typedef

Joined: 25 Jul 2010
Posts: 2909
Location: 0x77760000

typedef 12 Mar 2015, 19:19

system error wrote:

l_inc wrote:
system error

Quote:
...with very little reference sadly

Look here, here, here and here. I hope, that's enough reference.

LOL. Where do you think I learned creating my 32-bit DLL from?
But some of these don't work quite well in my 64-bit library.

One more question;

Since I don't have the program with me right now, I can't test whether I should use ExitProcess or just exit from msvcrt? It works fine in the 32-bit version. But I am not very sure anymore for the 64-bit version because this is a kind of cross-call between win64 and console. Could that be one of the problem?

raise(SIGTERM) Very Happy

12 Mar 2015, 19:19

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland

Tomasz Grysztar 12 Mar 2015, 20:04

I should probably include a 64-bit DLL example in the fasmw package. Here is a simple one that I just tested on Windows 8.1 (I worked from your code, correcting everything I thought could be corrected):

Code:

format PE64 console DLL
entry DllEntryPoint

include 'win64a.inc'

section '.text' code readable executable

proc DllEntryPoint hinstDLL,fdwReason,lpvReserved
        mov eax,TRUE
        ret
endp

proc TestMessage
        invoke  GetStdHandle,STD_OUTPUT_HANDLE
        mov     ebx,eax
        invoke  WriteFile,ebx,_test,_test.length,bytes_count,0
        invoke  CloseHandle,ebx
        ret
endp

section '.data' data readable

  _test db 'Test'
        .length = $ - _test

section '.bss' data readable writeable

  bytes_count dd ?

section '.edata' export data readable

    export 'thelib64.dll',\
        TestMessage,'TestMessage'

section '.reloc' fixups data readable discardable

    if $=$$
        dd 0,8                          ; if there are no fixups, generate dummy entry
    end if

section '.idata' import data readable writeable

  library kernel32,'kernel32.dll'

  include 'api/kernel32.inc'

If you prefer smaller file and less sections, instead of making a dummy fixups entry you can remove '.reloc' section altogether and add "data fixups" to some other section, for instance they can share a section with export information:

Code:

section '.edata' data readable

    data export
        export 'thelib64.dll',\
            TestMessage,'TestMessage'
    end data

    data fixups
    end data

12 Mar 2015, 20:04

l_inc

Joined: 23 Oct 2009
Posts: 881

l_inc 12 Mar 2015, 20:42

system error

Quote:

I did in 32-bit. Works.

It would work anyway, because import function calls in 32-bit code are done with absolute addressing, which requires fixups and hence automatically makes the section not empty. You'd know that if you followed the links I provided.

Quote:

I did in 64-bit. FAIL.

You did it wrong. You'd know that if you followed the links I provided.

_________________
Faith is a superposition of knowledge and fallacy

12 Mar 2015, 20:42

system error

Joined: 01 Sep 2013
Posts: 670

system error 13 Mar 2015, 11:52

Tomasz Grysztar wrote:

Code:

format PE64 console DLL
entry DllEntryPoint

include 'win64a.inc'

section '.text' code readable executable

proc DllEntryPoint hinstDLL,fdwReason,lpvReserved
        mov eax,TRUE
        ret
endp

proc TestMessage
        invoke  GetStdHandle,STD_OUTPUT_HANDLE
        mov     ebx,eax
        invoke  WriteFile,ebx,_test,_test.length,bytes_count,0
        invoke  CloseHandle,ebx
        ret
endp

section '.data' data readable

  _test db 'Test'
        .length = $ - _test

section '.bss' data readable writeable

  bytes_count dd ?

section '.edata' export data readable

    export 'thelib64.dll',\
        TestMessage,'TestMessage'

section '.reloc' fixups data readable discardable

    if $=$$
        dd 0,8                          ; if there are no fixups, generate dummy entry
    end if

section '.idata' import data readable writeable

  library kernel32,'kernel32.dll'

  include 'api/kernel32.inc'

Code:

section '.edata' data readable

    data export
        export 'thelib64.dll',\
            TestMessage,'TestMessage'
    end data

    data fixups
    end data

This is beautiful. A small piece of useful code from the creator himself.

13 Mar 2015, 11:52

masm

Joined: 10 Oct 2014
Posts: 6

masm 10 Mar 2020, 10:16

Is This exemple compatible with Windows 10

10 Mar 2020, 10:16

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum