simple 64-bit window not simple :(

Index > Windows > simple 64-bit window not simple :(

Author

Thread

magicSqr

Joined: 27 Aug 2011
Posts: 105

magicSqr 29 Jun 2015, 17:55

Hi,

I've been over this 'simple' code with a fine-tooth comb and I'm damned if I can see what is wrong. Using WinDBG I know that everything up to CreateWindowEx works. The CreateWindowEx fails. It then jumps to showError as it should. GetLastError and FormatMessage work but MessageBox doesn't return. Peeking at [errStrBuff] after FormatMessage, the error being returned is Invalid Window Handle?

Any idea why this simple 64-bit window isn't simple?

Thanks

Code:

format PE64 GUI 5.0
entry start

include "%fasminc%\win64ax.inc"

section '.code' code readable executable

    start:
        invoke  GetModuleHandle, 0
        mov         [wc.hInstance], rax
        invoke  LoadIcon, 0, IDI_APPLICATION
        mov         [wc.hIcon], rax
        invoke  LoadCursor, 0, IDC_ARROW
        mov         [wc.hCursor], rax
        invoke  RegisterClass, wc
        or          rax, rax
        jz          showError
        invoke  CreateWindowEx, NULL, myClass, titleStr, WS_OVERLAPPEDWINDOW or WS_VISIBLE, 100, 100, 100, 100, NULL, NULL, [wc.hInstance], NULL
        test    rax, rax
        jz          showError
        mov     [hWndMain], rax

    msgLoop:
        invoke  GetMessage, msg, NULL, 0, 0
        or          rax, rax
        jz          endLoop
        invoke  TranslateMessage, msg
        invoke  DispatchMessage, msg
        jmp         msgLoop
    showError: 
        invoke  GetLastError 
        invoke  FormatMessage, FORMAT_MESSAGE_FROM_SYSTEM or FORMAT_MESSAGE_ALLOCATE_BUFFER, NULL, rax, LANG_NEUTRAL, errStrBuff, 0, NULL
        invoke  MessageBox, HWND_DESKTOP, [errStrBuff], NULL, MB_ICONERROR or MB_OK 
    endLoop:
        invoke  ExitProcess, [msg.wParam]

proc WndProc hwnd, wmsg, wparam, lparam
        push    rbx rsi rdi
        cmp     [wmsg], WM_DESTROY
        je      .wmDestroy
    .defWndProc:
        invoke  DefWindowProc, [hwnd], [wmsg], [wparam], [lparam]
        jmp     .finish
    .wmDestroy:
        invoke  PostQuitMessage, 0
        xor         rax, rax
    .finish:
        pop         rdi rsi rbx
        ret
endp

section '.data' data readable writeable
    myClass     db "myWin", 00
    titleStr    db "Simple Window", 00
    hWndMain    dq ?
    wc          WNDCLASS CS_HREDRAW or CS_VREDRAW, WndProc, 0, 0, NULL, NULL, NULL, COLOR_BTNFACE + 1, NULL, myClass
    msg         MSG
    errStrBuff  dq ?

section '.idata' import data readable writeable 
library kernel32,'KERNEL32.DLL',\
        user32,  'USER32.DLL'

include '%fasminc%\api\Kernel32.inc' 
include '%fasminc%\api\User32.inc'

29 Jun 2015, 17:55

magicSqr

Joined: 27 Aug 2011
Posts: 105

magicSqr 29 Jun 2015, 21:03

ok, part of the problem was I didn't rtfm.

Parameters are passed to WndProc in the registers but there is still an anomoly.
Here's the new WndProc that does work...

Code:

proc WndProc hwnd, wmsg, wparam, lparam 
;        push    rbx rsi rdi 
        cmp     rdx, WM_DESTROY 
        je      .wmDestroy 
    .defWndProc: 
;        invoke  DefWindowProc, [hwnd], [wmsg], [wparam], [lparam] 
        invoke  DefWindowProc, rcx, rdx, r8, r9
        jmp     .finish 
    .wmDestroy: 
        invoke  PostQuitMessage, 0 
        xor         rax, rax 
    .finish: 
;        pop         rdi rsi rbx 
        ret 
endp

The strange part is, if I leave in the 'preserving' pushes and pops, the code doesn't work?? Anyone have any ideas?

Thanks again

29 Jun 2015, 21:03

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20462
Location: In your JS exploiting you and your system

revolution 29 Jun 2015, 21:46

You need to manually place the register parameters on to the top four places in the stack:

Code:

mov [hwnd],rcx

Else the values will be random.

29 Jun 2015, 21:46

magicSqr

Joined: 27 Aug 2011
Posts: 105

magicSqr 29 Jun 2015, 22:00

Thanks revolution, yes I know I can manually save the values of [hwnd] etc. That's not the problem, apparently I cannot preserve (be able to use and restore) register values since the

Code:

push    rbx rsi rdi
.
.
.
.
pop     rdi rsi rbx

stops my WndProc from working?

29 Jun 2015, 22:00

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20462
Location: In your JS exploiting you and your system

revolution 29 Jun 2015, 22:26

Unaligned stack. Push a multiple of 16 bytes at a time.

29 Jun 2015, 22:26

l_inc

Joined: 23 Oct 2009
Posts: 881

l_inc 29 Jun 2015, 22:32

magicSqr

Quote:

ok, part of the problem was I didn't rtfm.

It's not just a part, it's the whole problem. The stack must be 16 bytes aligned before making a call. The proc macro tries to ensure that, but your manual pushing corrupts the alignment. Let the proc macro do its job and specify the keyword uses followed by a list of registers:

Code:

proc WndProc uses rbx rsi rdi, hwnd, wmsg, wparam, lparam

_________________
Faith is a superposition of knowledge and fallacy

29 Jun 2015, 22:32

magicSqr

Joined: 27 Aug 2011
Posts: 105

magicSqr 29 Jun 2015, 23:10

l_inc wrote:

magicSqr

Quote:
ok, part of the problem was I didn't rtfm.

It's not just a part, it's the whole problem. The stack must be 16 bytes aligned before making a call. The proc macro tries to ensure that, but your manual pushing corrupts the alignment. Let the proc macro do its job and specify the keyword uses followed by a list of registers:
Code:
proc WndProc uses rbx rsi rdi, hwnd, wmsg, wparam, lparam    

ah!

Thanks for the clarification l_inc. At least I now know the problem was of my own making. Life was so much easier with 32-bit but I need 64 for this particular problem.

Thanks again Very Happy

29 Jun 2015, 23:10

magicSqr

Joined: 27 Aug 2011
Posts: 105

magicSqr 01 Jul 2015, 18:07

Hi again revolution and l_inc,

The code below is working now but there *appears* to be a leak. My reason for thinking this is that if I have Windows Task Manager running then open and close my window it is taking about 5 seconds to disappear from the Task Manager, the fasm example template.exe disappears almost instantaneously. I've tried it many times with the same result. Is there anything else wrong with my code?

Many thanks

magic²

Code:

format PE64 GUI 5.0 
entry start 

include "%fasminc%\win64a.inc" 

section '.code' code readable executable 

    start: 
        invoke  GetModuleHandle, 0 
        mov     [wc.hInstance], rax 
        invoke  LoadIcon, 0, IDI_APPLICATION 
        mov     [wc.hIcon], rax 
        invoke  LoadCursor, 0, IDC_ARROW 
        mov     [wc.hCursor], rax 
        invoke  RegisterClass, wc 
        or      rax, rax 
        jz      showError 
        invoke  CreateWindowEx, NULL, myClass, titleStr, WS_OVERLAPPEDWINDOW or WS_VISIBLE, 100, 100, 600, 400, NULL, NULL, [wc.hInstance], NULL
        or      rax, raX
        jz      showError 
        mov     [hWndMain], rax 

    msgLoop: 
        invoke  GetMessage, msg, NULL, 0, 0 
        or      rax, rax 
        jz      endLoop 
        invoke  TranslateMessage, msg 
        invoke  DispatchMessage, msg 
        jmp     msgLoop 
    showError:  
        invoke  GetLastError  
        invoke  FormatMessage, FORMAT_MESSAGE_FROM_SYSTEM or FORMAT_MESSAGE_ALLOCATE_BUFFER, NULL, rax, LANG_NEUTRAL, errStrBuff, 0, NULL
        invoke  MessageBox, HWND_DESKTOP, [errStrBuff], NULL, MB_ICONERROR or MB_OK  
    endLoop: 
        invoke  ExitProcess, [msg.wParam] 

proc WndProc uses rbx rdi rsi, hwnd, wmsg, wparam, lparam            
        cmp     edx, WM_DESTROY                                   
        je      .wmDestroy                                           
    .defWndProc:                                                     
        invoke  DefWindowProc, rcx, rdx, r8, r9     ;[hwnd], [wmsg], [wparam], [lparam]    
        jmp     .finish                                              
    .wmDestroy:                                                      
        invoke  PostQuitMessage, 0                                   
        xor         rax, rax                                         
    .finish:                                                         
        ret                                                          
endp                                                                 
                                                                     
section '.data' data readable writeable 
    myClass     db "myWin", 00 
    titleStr    db "Simple Window", 00 
    hWndMain    dq ? 
    wc          WNDCLASS CS_HREDRAW or CS_VREDRAW, WndProc, 0, 0, NULL, NULL, NULL, COLOR_BTNFACE + 1, NULL, myClass 
    msg         MSG 
    errStrBuff  dq ? 

section '.idata' import data readable writeable  

    library kernel32,'KERNEL32.DLL',\ 
            user32,  'USER32.DLL' 

    include '%fasminc%\api\Kernel32.inc'  
    include '%fasminc%\api\User32.inc'

01 Jul 2015, 18:07

typedef

Joined: 25 Jul 2010
Posts: 2909
Location: 0x77760000

typedef 01 Jul 2015, 18:30

Lol. Basic Windows APIs and yet we all have problems with them.

01 Jul 2015, 18:30

magicSqr

Joined: 27 Aug 2011
Posts: 105

magicSqr 01 Jul 2015, 18:56

typedef wrote:

Lol. Basic Windows APIs and yet we all have problems with them.

You're not wrong typedef Laughing

01 Jul 2015, 18:56

typedef

Joined: 25 Jul 2010
Posts: 2909
Location: 0x77760000

typedef 01 Jul 2015, 21:45

WNDCLASSEXW = CreateWindowExW
WNDCLASSEXA = CreateWindowExA

You also need to call the RegisterClass API

01 Jul 2015, 21:45

l_inc

Joined: 23 Oct 2009
Posts: 881

l_inc 01 Jul 2015, 22:36

magicSqr
The problem is the same as before: unaligned stack. Your entry point from the system's point of view is nothing but a function, which means that the call to it follows the same stack alignment constraints: 16 bytes aligned before the call. But the call itself puts 8 bytes of the return address onto the stack. For that reason you're supposed to care about reestablishing the alignment before making API calls. Just put sub rsp,8 at your entry point.

Additionally I'd suggest you to either put string constants at the end of the data section or to put at least align 8 in front of qword-sized variables. Otherwise you might experience undesired effects as well.

_________________
Faith is a superposition of knowledge and fallacy

01 Jul 2015, 22:36

Mikl___

Joined: 30 Dec 2014
Posts: 143
Location: Russian Federation, Irkutsk

Mikl___ 02 Jul 2015, 00:34

Hi, magicSqr!

Code:

format PE64 GUI 5.0
entry WinMain
include 'win64a.inc'


section '.text' code readable writeable executable

_class TCHAR "The simplest window in FASM",0 ;name of our window and name of class
proc WinMain
IMAGE_BASE = $-rva $
local msg:MSG
          ; +------------------------------+ 
          ; | registering the window class | 
          ; +------------------------------+
          xor ebx,ebx
          mov edi,_class
          mov esi,IMAGE_BASE
          mov eax,10027h
          push rax   ;hIconSm
          push rdi        ;lpszClassName
          push rbx        ;lpszMenuName
          push COLOR_WINDOW;hbrBackground
          push 10005h     ;hCursor
          push rax        ;hIcon
          push rsi        ;hInstance
          push rbx        ;cbClsExtra & cbWndExtra
          push WindowProc  ;lpfnWndProc
          push sizeof.WNDCLASSEX;cbSize & style
          mov ecx,esp     ;addr WNDCLASSEX
          call [RegisterClassEx]
          ; +--------------------------+
          ; | creating the main window |
          ; +--------------------------+
          push rbx
          push rsi        ;rsi=400000h
          shl esi,9       ;rsi=CW_USEDEFAULT
          push rbx
          push rbx
          push rsi
          push rsi
          push rsi
          push rsi
          mov r9d,WS_OVERLAPPEDWINDOW or WS_VISIBLE
          mov r8d,edi      ;offset ClassName
          mov edx,edi     ;offset ClassName
          xor ecx,ecx
          sub esp,20h
          call [CreateWindowEx]
          lea edi,[msg]
          ; +---------------------------+
          ; | entering the message loop |
          ; +---------------------------+
window_message_loop_start:
          mov ecx,edi
          xor edx,edx
          mov r8,rbx
          mov r9,rbx
          call [GetMessage]
          mov ecx,edi
          call [DispatchMessage]
          jmp  window_message_loop_start
endp
;---------------------------------------------
          ; +----------------------+
          ; | the window procedure |
          ; +----------------------+
WindowProc:
              cmp  edx,WM_DESTROY
              je   wmDESTROY
              jmp [DefWindowProc]
wmDESTROY:    xor ecx,ecx
              call [ExitProcess]
;---------------------------------------------------------------
data import
     library   KERNEL32, 'KERNEL32.DLL',\
               USER32,   'USER32.DLL'

     import    KERNEL32,\
               ExitProcess,        'ExitProcess'

     import    USER32,\
               RegisterClassEx,      'RegisterClassExA',\
               CreateWindowEx,     'CreateWindowExA',\
               DefWindowProc,      'DefWindowProcA',\
               GetMessage,         'GetMessageA',\
               DispatchMessage,    'DispatchMessageA'
end data

keep it short and simple!

02 Jul 2015, 00:34

magicSqr

Joined: 27 Aug 2011
Posts: 105

magicSqr 03 Jul 2015, 11:27

l_inc wrote:

magicSqr
The problem is the same as before: unaligned stack. Your entry point from the system's point of view is nothing but a function, which means that the call to it follows the same stack alignment constraints: 16 bytes aligned before the call. But the call itself puts 8 bytes of the return address onto the stack. For that reason you're supposed to care about reestablishing the alignment before making API calls. Just put sub rsp,8 at your entry point.

Additionally I'd suggest you to either put string constants at the end of the data section or to put at least align 8 in front of qword-sized variables. Otherwise you might experience undesired effects as well.

Thanks again l_inc. That has it all fixed now. I had written a good few 64-bit console programs without any issues, it was when I tried this simple GUI that I came unstuck. Oh well, it's a bad day when you don't learn something Wink

03 Jul 2015, 11:27

Mikl___

Joined: 30 Dec 2014
Posts: 143
Location: Russian Federation, Irkutsk

Mikl___ 08 Jul 2015, 05:36

magicSqr,
My solution of the problem "simple 64-bit window is not simple" you did not like?

Examples for Win64 Iczelion tutorial

08 Jul 2015, 05:36

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum