flat assembler
Message board for the users of flat assembler.

Index > Windows > Creating Console DLL 64-bit

Author
Thread Post new topic Reply to topic
system error



Joined: 01 Sep 2013
Posts: 671
system error
How to properly create a console-based PE64 DLL? I managed to make one for 32-bit but not in 64-bit Windows 8. I am using FASMW 1.71.34

My code structure looks something like this
Code:
;File: thelib64.asm
format PE64 console DLL
entry DllEntryPoint
include 'win64axp.inc'

section '.text' code readable executable

DllEntryPoint:
        sub rsp,8
        mov eax,1
        ret 24 ;?

;The other stuff

section '.edata' export data readable
        export 'thelib64.dll',\
       ;The other stuff names and aliases

section '.reloc' fixups data readable discardable ;switch this ON and OFF but still fail
;empty

section '.idata' import data readable
;some import stuff from msvcrt.dll     


I call it from this code;
Code:
;File: calldll.asm
format PE64 console
include 'win64axp.inc'
entry start

section '.text' code readable executable
start:
      
      mov rcx,[x]
      call [theOtherStuff]  
      call [exit]

section '.data' data readable writeable
x dq 34

section '.idata' import data readable
  library thelib64,'thelib64.dll'
  import thelib64,\
         ;the Other stuff import
    


The errors are either

1. ..."Not started correctly..."
2. ..."Not a win64 application... contact vendor bla bla..."

My questions;

1. What's wrong? The same code structure works perfectly in 32-bit version

2. Is it even possible to create a console program by calling win64 header like this, considering the same code structure working just fine in 32-bit version.
Post 12 Mar 2015, 13:20
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17717
Location: In your JS exploiting you and your system
revolution
I'm assuming you already used a search engine and came up with no answers. So based upon that assumption and your code above I'd suggest that a 64-bit console DLL is not supported.

Therefore, if you still need to have a console window for your DLL then you can use AllocConsole. But the new console won't be attached to the local console window.
Post 12 Mar 2015, 13:29
View user's profile Send private message Visit poster's website Reply with quote
system error



Joined: 01 Sep 2013
Posts: 671
system error
revolution wrote:
I'm assuming you already used a search engine and came up with no answers. So based upon that assumption and your code above I'd suggest that a 64-bit console DLL is not supported.

Therefore, if you still need to have a console window for your DLL then you can use AllocConsole. But the new console won't be attached to the local console window.


Most examples on the web shows GUI version. Too bad.

But the same code works for 32-bit version on the same windows 8 (64-bit). That's weird though.

Is there any code 'fix' for this case?
Post 12 Mar 2015, 13:34
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17717
Location: In your JS exploiting you and your system
revolution
If you don't need a separate console window for your DLL then just make the DLL a GUI. You can still link to GUI DLLs from a console application, everything will still work.
Post 12 Mar 2015, 13:41
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
revolution
Quote:
I'm assuming you already used a search engine

A very rarely justified assumption. He did not. It's the well known empty section problem.

_________________
Faith is a superposition of knowledge and fallacy
Post 12 Mar 2015, 13:42
View user's profile Send private message Reply with quote
system error



Joined: 01 Sep 2013
Posts: 671
system error
l_inc wrote:
It's the well known empty section problem.


...with very little reference sadly. Every web findings didn't really say it's a 'problem'. Some offer workaround to make the 'problem' disappear. But they all don't work in this particular case. That's why this thread. LOL.
Post 12 Mar 2015, 16:58
View user's profile Send private message Reply with quote
system error



Joined: 01 Sep 2013
Posts: 671
system error
revolution wrote:
If you don't need a separate console window for your DLL then just make the DLL a GUI. You can still link to GUI DLLs from a console application, everything will still work.
Didn't do the trick. If it works for 32-bit version on the same console, I don't see why 64-bit version shouldn't work. They are both on the same Win8 anyway. I am thinking that maybe I am missing some basic steps in creating 64-bit dll. I don't know.
Post 12 Mar 2015, 17:24
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
system error
Quote:
...with very little reference sadly

Look here, here, here and here. I hope, that's enough reference.

_________________
Faith is a superposition of knowledge and fallacy
Post 12 Mar 2015, 18:01
View user's profile Send private message Reply with quote
system error



Joined: 01 Sep 2013
Posts: 671
system error
l_inc wrote:
system error
Quote:
...with very little reference sadly

Look here, here, here and here. I hope, that's enough reference.


LOL. Where do you think I learned creating my 32-bit DLL from?
But some of these don't work quite well in my 64-bit library.

One more question;

Since I don't have the program with me right now, I can't test whether I should use ExitProcess or just exit from msvcrt? It works fine in the 32-bit version. But I am not very sure anymore for the 64-bit version because this is a kind of cross-call between win64 and console. Could that be one of the problem?
Post 12 Mar 2015, 18:12
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
system error
Quote:
LOL. Where do you think I learned creating my 32-bit DLL from?

No idea, but if you followed at least one suggestion to make your relocations section not empty, you wouldn't have the problem.
Quote:
this is a kind of cross-call between win64 and console

Oh, come on! GUI or console in a dll makes no difference. You can put any with no consequences.

_________________
Faith is a superposition of knowledge and fallacy
Post 12 Mar 2015, 18:18
View user's profile Send private message Reply with quote
system error



Joined: 01 Sep 2013
Posts: 671
system error
l_inc wrote:
No idea, but if you followed at least one suggestion to make your relocations section not empty, you wouldn't have the problem.


I did in 32-bit. Works.
I did in 64-bit. FAIL.

l_inc wrote:
Oh, come on! GUI or console in a dll makes no difference. You can put any with no consequences.


Really? Ok then Very Happy
Post 12 Mar 2015, 18:29
View user's profile Send private message Reply with quote
typedef



Joined: 25 Jul 2010
Posts: 2913
Location: 0x77760000
typedef
system error wrote:
l_inc wrote:
system error
Quote:
...with very little reference sadly

Look here, here, here and here. I hope, that's enough reference.


LOL. Where do you think I learned creating my 32-bit DLL from?
But some of these don't work quite well in my 64-bit library.

One more question;

Since I don't have the program with me right now, I can't test whether I should use ExitProcess or just exit from msvcrt? It works fine in the 32-bit version. But I am not very sure anymore for the 64-bit version because this is a kind of cross-call between win64 and console. Could that be one of the problem?


raise(SIGTERM) Very Happy
Post 12 Mar 2015, 19:19
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7802
Location: Kraków, Poland
Tomasz Grysztar
I should probably include a 64-bit DLL example in the fasmw package. Here is a simple one that I just tested on Windows 8.1 (I worked from your code, correcting everything I thought could be corrected):
Code:
format PE64 console DLL
entry DllEntryPoint

include 'win64a.inc'

section '.text' code readable executable

proc DllEntryPoint hinstDLL,fdwReason,lpvReserved
        mov eax,TRUE
        ret
endp

proc TestMessage
        invoke  GetStdHandle,STD_OUTPUT_HANDLE
        mov     ebx,eax
        invoke  WriteFile,ebx,_test,_test.length,bytes_count,0
        invoke  CloseHandle,ebx
        ret
endp

section '.data' data readable

  _test db 'Test'
        .length = $ - _test

section '.bss' data readable writeable

  bytes_count dd ?

section '.edata' export data readable

    export 'thelib64.dll',\
        TestMessage,'TestMessage'

section '.reloc' fixups data readable discardable

    if $=$$
        dd 0,8                          ; if there are no fixups, generate dummy entry
    end if

section '.idata' import data readable writeable

  library kernel32,'kernel32.dll'

  include 'api/kernel32.inc'

    
If you prefer smaller file and less sections, instead of making a dummy fixups entry you can remove '.reloc' section altogether and add "data fixups" to some other section, for instance they can share a section with export information:
Code:
section '.edata' data readable

    data export
        export 'thelib64.dll',\
            TestMessage,'TestMessage'
    end data

    data fixups
    end data    
Post 12 Mar 2015, 20:04
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
system error
Quote:
I did in 32-bit. Works.

It would work anyway, because import function calls in 32-bit code are done with absolute addressing, which requires fixups and hence automatically makes the section not empty. You'd know that if you followed the links I provided.
Quote:
I did in 64-bit. FAIL.

You did it wrong. You'd know that if you followed the links I provided.

_________________
Faith is a superposition of knowledge and fallacy
Post 12 Mar 2015, 20:42
View user's profile Send private message Reply with quote
system error



Joined: 01 Sep 2013
Posts: 671
system error
Tomasz Grysztar wrote:
I should probably include a 64-bit DLL example in the fasmw package. Here is a simple one that I just tested on Windows 8.1 (I worked from your code, correcting everything I thought could be corrected):
Code:
format PE64 console DLL
entry DllEntryPoint

include 'win64a.inc'

section '.text' code readable executable

proc DllEntryPoint hinstDLL,fdwReason,lpvReserved
        mov eax,TRUE
        ret
endp

proc TestMessage
        invoke  GetStdHandle,STD_OUTPUT_HANDLE
        mov     ebx,eax
        invoke  WriteFile,ebx,_test,_test.length,bytes_count,0
        invoke  CloseHandle,ebx
        ret
endp

section '.data' data readable

  _test db 'Test'
        .length = $ - _test

section '.bss' data readable writeable

  bytes_count dd ?

section '.edata' export data readable

    export 'thelib64.dll',\
        TestMessage,'TestMessage'

section '.reloc' fixups data readable discardable

    if $=$$
        dd 0,8                          ; if there are no fixups, generate dummy entry
    end if

section '.idata' import data readable writeable

  library kernel32,'kernel32.dll'

  include 'api/kernel32.inc'

    
If you prefer smaller file and less sections, instead of making a dummy fixups entry you can remove '.reloc' section altogether and add "data fixups" to some other section, for instance they can share a section with export information:
Code:
section '.edata' data readable

    data export
        export 'thelib64.dll',\
            TestMessage,'TestMessage'
    end data

    data fixups
    end data    


This is beautiful. A small piece of useful code from the creator himself.
Post 13 Mar 2015, 11:52
View user's profile Send private message Reply with quote
masm



Joined: 10 Oct 2014
Posts: 6
masm
Is This exemple compatible with Windows 10
Post 10 Mar 2020, 10:16
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on GitHub, YouTube, Twitter.

Website powered by rwasa.