How to click a button?

Index > Windows > How to click a button?

Goto page 1, 2 Next

Author

Thread

semiono

Joined: 31 Aug 2007
Posts: 198
Location: section '.code' executable

semiono 03 Jan 2025, 15:09

Code:

include '%fasm%\win64ax.inc'
 
start:
        sub rsp,8
 
        invoke ShellExecute,NULL,NULL,'calc.exe',NULL,NULL,SW_RESTORE
        invoke Sleep,500
 
        invoke FindWindow,'CalcFrame',NULL
        mov [i],eax
        invoke FindWindowEx,[i],NULL,NULL,'3'
        mov [o],eax
        invoke SendMessage,[o],BM_CLICK,NULL,NULL
exit:
        invoke ExitProcess,NULL
 
section '.data' readable writeable
 
        i dd NULL
        o dd NULL
 
section '.idata' import readable
 
        library kernel32,'KERNEL32.DLL',shell32,'SHELL32.DLL',user32,'USER32.DLL'
        include '%fasm%/api/kernel32.inc'
        include '%fasm%/api/shell32.inc'
        include '%fasm%/api/user32.inc'

Not clickable? BTW How click a ListBox item?

Smile

_________________
Windows 9, FL Studio 19

03 Jan 2025, 15:09

macomics

Joined: 26 Jan 2021
Posts: 1041
Location: Russia

macomics 03 Jan 2025, 18:24

semiono wrote:

BTW How click a ListBox item?

Use LBN_SELCHANGE or LBN_DBLCLK for this.

semiono wrote:

Not clickable?

FindWindowEx does not find the button with the title '3' because the calculator has changed the design of the form elements.

To search for button, you should use EnumChildWindows and check the additional parameters of the listed windows to distinguish the various form elements from each other.

03 Jan 2025, 18:24

semiono

Joined: 31 Aug 2007
Posts: 198
Location: section '.code' executable

semiono 04 Jan 2025, 00:48

This is a friend)

04 Jan 2025, 00:48

Mat Quasar

Joined: 15 Dec 2024
Posts: 54

Mat Quasar 04 Jan 2025, 08:51

I tried to use this piece of code, by referring to C++ code from StackOverflow ( https://stackoverflow.com/questions/1223872/programmatically-press-a-button-on-another-application-c-windows ):

It doesn't work either, the callback of EnumChildWindows not receiving control?

Code:

format PE GUI
include 'win32a.inc'

section '.code' code readable executable

entry $

  invoke    FindWindow,0,_title
  mov       [_window], eax
  or        eax, eax
  jz        .quit
  invoke    MessageBox,0,_message2,_title,MB_OK          ;'Calculator' window found
  invoke    EnumChildWindows,[_window],EnumChildWindowsProc,0
  cmp       [_handle],0
  jz        .quit
  invoke    MessageBox,0,_message,_title,MB_OK           ;'3' button found
  invoke    SendMessage,[_handle],BM_CLICK,0,0
  ;invoke    SendMessage,[_handle],WM_LBUTTONDOWN,0,0
.quit:
  invoke    ExitProcess,0

proc EnumChildWindowsProc,hwnd,lparam
  push      ebx esi edi

  invoke    GetWindowText,[hwnd],_buffer,100
  or        eax,eax
  jz        .fail

  invoke    lstrcmp,_button,_buffer
  or        eax,eax
  jnz       .fail
  mov       ebx, [hwnd]
  mov       [_handle], ebx
  jmp       .succeed

.fail:
  mov       eax, -1

.succeed:
  pop       edi esi ebx
  ret
endp

section '.data' data readable writeable

  _title      db 'Calculator',0
  _class      db 'CalcFrame',0
  _button     db '3',0          ;or '&3'
  _message    db 'Button found!',0
  _message2   db 'Looking for button...',0
  _handle     dd 0
  _buffer     rb 100
  _window     dd ?

section '.idata' import data readable writable

  library kernel32,'KERNEL32.DLL',\
          user32,'USER32.DLL',\
          shell32, 'SHELL32.DLL'

  import kernel32,\
         lstrcmp,'lstrcmpA',\
         Sleep,'Sleep',\
         Beep, 'Beep',\
         ExitProcess,'ExitProcess'

  import user32,\
         EnumChildWindows,'EnumChildWindows',\
         FindWindowEx,'FindWindowExA',\
         FindWindow, 'FindWindowA',\
         MessageBox,'MessageBoxA',\
         SendMessage,'SendMessageA',\
         GetWindowText,'GetWindowTextA'

  import shell32, \
         ShellExecute,'ShellExecuteA'

Can someone help?

04 Jan 2025, 08:51

Core i7

Joined: 14 Nov 2024
Posts: 28
Location: Socket in motherboard

Core i7 04 Jan 2025, 09:48

Mat Quasar wrote:

Can someone help?

Previously, to support the window interface, Windows used WinForms technology, which was replaced in Win8 with the WPF framework, in Win10 with UWP, and finally in Win11 with WinUI. Thus, the entire system interface (including the calculator) is written on the corresponding frameworks, and EnumChildWindows() is designed only for the outdated WinForms - therefore, this function for system applications does not work on Win8+, but only on WinXP and Seven.

04 Jan 2025, 09:48

Mat Quasar

Joined: 15 Dec 2024
Posts: 54

Mat Quasar 04 Jan 2025, 11:11

I see, thanks for the clarification, Core i7.

04 Jan 2025, 11:11

Mat Quasar

Joined: 15 Dec 2024
Posts: 54

Mat Quasar 04 Jan 2025, 11:21

Tested, my code worked on Windows XP only, not on Windows 10.

Description:	Button "3" clicked after closing the message box.
Filesize:	65.6 KB
Viewed:	831 Time(s)

04 Jan 2025, 11:21

AsmGuru62

Joined: 28 Jan 2004
Posts: 1671
Location: Toronto, Canada

AsmGuru62 04 Jan 2025, 13:22

It always was curious for me.
Basically, with SendMessage across processes, you can 'click' stuff in other programs.
As usual, every technology advancement, well, almost every, can be abused, of course.
So, you can click on programs in a mallicious kind of way (and that is why current Windows OS takes that away).
But, you also can write a program to click and TEST other software, so a person who does testing is not as busy and can do other stuff.
Currently, is there a way to TEST programs (just by remotely clicking) as it was in XP?
Or, the testing by "clicking" quietly went away?
Anyone knows?

04 Jan 2025, 13:22

Mat Quasar

Joined: 15 Dec 2024
Posts: 54

Mat Quasar 04 Jan 2025, 13:47

Maybe SendInput Win32 API can?

https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendinput

There is an C++ example which I want to convert to FASM, but I don't know how to translate the INPUT structure.

EDIT: Wait, I am doing it...

04 Jan 2025, 13:47

macomics

Joined: 26 Jan 2021
Posts: 1041
Location: Russia

macomics 04 Jan 2025, 14:28

Code:

; typedef struct tagMOUSEINPUT {
;   LONG      dx;
;   LONG      dy;
;   DWORD     mouseData;
;   DWORD     dwFlags;
;   DWORD     time;
;   ULONG_PTR dwExtraInfo;
; } MOUSEINPUT, *PMOUSEINPUT, *LPMOUSEINPUT;

; typedef struct tagKEYBDINPUT {
;   WORD      wVk;
;   WORD      wScan;
;   DWORD     dwFlags;
;   DWORD     time;
;   ULONG_PTR dwExtraInfo;
; } KEYBDINPUT, *PKEYBDINPUT, *LPKEYBDINPUT;

; typedef struct tagHARDWAREINPUT {
;   DWORD uMsg;
;   WORD  wParamL;
;   WORD  wParamH;
; } HARDWAREINPUT, *PHARDWAREINPUT, *LPHARDWAREINPUT;

; typedef struct tagINPUT {
;   DWORD type;
;   union {
;     MOUSEINPUT    mi;
;     KEYBDINPUT    ki;
;     HARDWAREINPUT hi;
;   } DUMMYUNIONNAME;
; } INPUT, *PINPUT, *LPINPUT;

struc UNION base*, type& { if ~ type eq
    virtual at base
      . type
      if ~ definnite .Length
        label .Length at $ - $$
      end if
    end virtual
  else
    label . at base
    label .Length at 0
  end if
}

struc MOUSEINPUT32 {
  .dx dd ?
  .dy dd ?
  .mouseData dd ?
  .dwFlags dd ?
  .time dd ?
  .dwExtraInfo dd ?
}

struc MOUSEINPUT64 {
  .dx dd ?
  .dy dd ?
  .mouseData dd ?
  .dwFlags dd ?
  .time dd ?, ?
  .dwExtraInfo dq ?
}

struc KEYBOARDINPUT32 {
  .wVk dw ?
  .wScan dw ?
  .dwFlags dd ?
  .time dd ?
  .dwExtraInfo dd ?
}

struc KEYBOARDINPUT64 {
  .wVk dw ?
  .wScan dw ?
  .dwFlags dd ?
  .time dd ?, ?
  .dwExtraInfo dq ?
}

struc HARDWAREINPUT {
  .uMsg dd ?
  .wParamL dw ?
  .wParamH dw ?
}

struc INPUT32 {
  .type dd ?
  .mi MOUSEINPUT32 ; longest struc to create a buffer for the correct amount of data
  .ki UNION .mi, KEYBOARDINPUT32 ; These lines only create labels relative to .mi
  .hi UNION .mi, HARDWAREINPUT
}

struc INPUT64 {
  .type dd ?
  .mi MOUSEINPUT64 ; longest struc to create a buffer for the correct amount of data
  .ki UNION .mi, KEYBOARDINPUT64 ; These lines only create labels relative to .mi
  .hi UNION .mi, HARDWAREINPUT
}

Last edited by macomics on 04 Jan 2025, 14:32; edited 1 time in total

04 Jan 2025, 14:28

Mat Quasar

Joined: 15 Dec 2024
Posts: 54

Mat Quasar 04 Jan 2025, 14:30

This works in even Windows 10.....

Code:

format PE GUI
include 'win32a.inc'

INPUT_MOUSE    = 0
INPUT_KEYBOARD = 1
INPUT_HARDWARE = 2

struct INPUT
  .type         dd       ?       ;mouse=0, keyboard=1
  .wVk          dw       ?
                dw       ?
  .wScan        dw       ?
                dw       ?
  .dwFlags      dd       ?
  .time         dd       ?
  .dwExtraInfo  dq       ?
ends
align 8

section '.code' code readable executable

entry $

  invoke    FindWindow,0,_title
  or        eax, eax
  jz        .quit
  invoke    SetForegroundWindow, eax
  invoke    Sleep, 500
  invoke    SendInput, 2, _input, sizeof.INPUT
  or        eax,eax
  jz        .quit
  ;invoke    MessageBox, 0, _message, _title, MB_OK

.quit:
  invoke    ExitProcess,0

section '.data' data readable writeable

  _title      db 'Calculator',0
  _message    db 'Key "3" pressed',0
  _input      INPUT INPUT_KEYBOARD,'3',0,0,0,0
              INPUT INPUT_KEYBOARD,'3',0,KEYEVENTF_KEYUP,0,0

align 8

section '.idata' import data readable writable

  library kernel32,'KERNEL32.DLL',\
          user32,'USER32.DLL'

  import kernel32,\
         Sleep,'Sleep',\
         ExitProcess,'ExitProcess'

  import user32,\
         MessageBox, 'MessageBoxA',\
         FindWindow, 'FindWindowA',\
         SendInput,'SendInput',\
         SetForegroundWindow,'SetForegroundWindow'

04 Jan 2025, 14:30

Mat Quasar

Joined: 15 Dec 2024
Posts: 54

Mat Quasar 04 Jan 2025, 14:32

Thank you , macomics. However, I have finished doing the program using a little tweaked structure.

04 Jan 2025, 14:32

AsmGuru62

Joined: 28 Jan 2004
Posts: 1671
Location: Toronto, Canada

AsmGuru62 04 Jan 2025, 15:57

Awesome!

04 Jan 2025, 15:57

semiono

Joined: 31 Aug 2007
Posts: 198
Location: section '.code' executable

semiono 04 Jan 2025, 17:32

Mat Quasar wrote:

This works in even Windows 10.....

This works in even Windows 8.1

Very Happy

_________________
Windows 9, FL Studio 19

04 Jan 2025, 17:32

Mat Quasar

Joined: 15 Dec 2024
Posts: 54

Mat Quasar 04 Jan 2025, 19:55

AsmGuru62 wrote:

Awesome!

semiono wrote:

Mat Quasar wrote:
This works in even Windows 10.....

This works in even Windows 8.1

Thank you.

04 Jan 2025, 19:55

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 05 Jan 2025, 06:41

Universal Windows Platform (UWP) applications can be more involved, but we can use UI Automation to programatically interact with the Windows 11 calculator application in the following way ...

Code:

        virtual at rbp-.local
                .pAutomation    IUIAutomation
                .pCalculator    IUIAutomationElement
                .pCondition     IUIAutomationCondition
                .pButton3       IUIAutomationElement
                .pInvoke        IUIAutomationInvokePattern
                .var            VARIANT
                .local := ($ - $$) and -16
        end virtual
        enter .frame+.local, 0
        CoInitialize NULL
        CoCreateInstance & CLSID_CUIAutomation, NULL, CLSCTX_INPROC_SERVER,\
                & IID_IUIAutomation, & .pAutomation

        FindWindowA A 'ApplicationFrameWindow', A 'Calculator'
        xchg rdx, rax
        IUIAutomation__ElementFromHandle [.pAutomation], rdx, & .pCalculator
        cmp [.pCalculator], 0
        jz .no_element

        SysAllocString W "num3Button"
        mov [.var.qVal], rax
        mov [.var.vt], VT_BSTR
        IUIAutomation__CreatePropertyCondition [.pAutomation], \
                UIA_AutomationIdPropertyId, & .var, & .pCondition
        IUIAutomationElement__FindFirst [.pCalculator],\
                TreeScope_Descendants, [.pCondition], & .pButton3
        cmp [.pButton3], 0
        jz .no_condition_element

        IUIAutomationElement__GetCurrentPattern [.pButton3],\
                UIA_InvokePatternId, & .pInvoke
        cmp [.pInvoke], 0
        jz .no_pattern
        IUIAutomationInvokePattern__Invoke [.pInvoke]
        IUIAutomationInvokePattern__Release [.pInvoke]
.no_pattern:
        IUIAutomationElement__Release [.pButton3]
.no_condition_element:
        IUIAutomationCondition__Release [.pCondition]
        IUIAutomationElement__Release [.pCalculator]
        SysFreeString [.var.qVal] ; VariantClear & .var ; also works
.no_element:
        IUIAutomation__Release [.pAutomation]

        CoUninitialize

... this should work with WinUI applications as well. Use the inspect.exe tool, which comes with the Windows SDK, to explore automation support that applications have.

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

05 Jan 2025, 06:41

Mat Quasar

Joined: 15 Dec 2024
Posts: 54

Mat Quasar 05 Jan 2025, 11:42

bitRAKE wrote:

Use the inspect.exe tool, which comes with the Windows SDK, to explore automation support that applications have.

Just found it in my Windows SDK folder, it displayed the "ApplicationFrameWindow" class name and "num3button" Automation Id of Calculator app.

I am amazed by your fasm2/fasmg toolchain (which is linkable) in building sophisticated programs.
Nice work!

ADD: Your uiauto.exe is working for Calculator app in Windows 10.

Description:
Filesize:	99.78 KB
Viewed:	566 Time(s)

05 Jan 2025, 11:42

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 05 Jan 2025, 14:51

Mat Quasar wrote:

I am amazed by your fasm2/fasmg toolchain (which is linkable) in building sophisticated programs.
Nice work!

ADD: Your uiauto.exe is working for Calculator app in Windows 10.

Thank you for testing.

I've been trying to implement much type checking - to reduce the frailty of the layers. It's a major refactoring that converts the IDL files from the SDK. In the long-term the complexity will prevent anyone from using the toolchain unless usage is more robust, imho.

The COM interfaces support three different syntaxes - the above syntax is the middle verbosity, but the terse version is (C++ like, imho):

Code:

        virtual at rbp-.local
                .pAutomation    IUIAutomation
                .pCalculator    IUIAutomationElement
                .pCondition     IUIAutomationCondition
                .pButton3       IUIAutomationElement
                .pInvoke        IUIAutomationInvokePattern
                .var            VARIANT
                .local := ($ - $$) and -16
        end virtual
        enter .frame+.local, 0
        CoInitialize NULL
        CoCreateInstance & CLSID_CUIAutomation, NULL, CLSCTX_INPROC_SERVER,\
                & IID_IUIAutomation, & .pAutomation

        FindWindowA A 'ApplicationFrameWindow', A 'Calculator'
        xchg rdx, rax
        .pAutomation->ElementFromHandle rdx, & .pCalculator
        cmp [.pCalculator], 0
        jz .no_element

        SysAllocString W "num3Button"
        mov [.var.qVal], rax
        mov [.var.vt], VT_BSTR
        .pAutomation->CreatePropertyCondition UIA_AutomationIdPropertyId, & .var, & .pCondition
        .pCalculator->FindFirst TreeScope_Descendants, [.pCondition], & .pButton3
        cmp [.pButton3], 0
        jz .no_condition_element

        .pButton3->GetCurrentPattern UIA_InvokePatternId, & .pInvoke
        cmp [.pInvoke], 0
        jz .no_pattern
        .pInvoke->Invoke
        .pInvoke->Release
.no_pattern:
        .pButton3->Release
.no_condition_element:
        .pCondition->Release
        .pCalculator->Release
        SysFreeString [.var.qVal] ; VariantClear & .var ; also works
.no_element:
        .pAutomation->Release

        CoUninitialize

... stating the object type just the once and of course the fully manual syntax - which look closest to what is seen in the debugger.

Code:

        mov rcx, [.irich]
        mov rax, [rcx]
        call [rax + IUnknown__Release]

All these little tests help work through usage problems, complexities with other fasmg language features, etc. And I take notes on all my confusions as I improve my understanding.

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

05 Jan 2025, 14:51

semiono

Joined: 31 Aug 2007
Posts: 198
Location: section '.code' executable

semiono 07 Jan 2025, 23:31

Please, an example with ListBox the same)
Focus, scroll and click item.

This close all tasks in practice reason)

Smile

07 Jan 2025, 23:31

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 08 Jan 2025, 15:59

The pseudo-code would look something like ...

Code:

; Find list box
; (Assume .pListBox is your IUIAutomationElement pointer)

; 1) Focus:
        .pListBox->SetFocus

; 2) Scroll:
        .pListBox->GetCurrentPattern UIA_ScrollPatternId, &.pScroll
        cmp [.pScroll], 0
        jz .no_scroll_pattern
        .pScroll->Scroll ScrollAmount_LargeIncrement, ScrollAmount_NoAmount
        .pScroll->Release
.no_scroll_pattern:

; 3) Find and select list item:
        SysAllocString W "MyListItem"              ; Or find by other property
        mov [.var.qVal], rax
        mov [.var.vt], VT_BSTR
        .pAutomation->CreatePropertyCondition UIA_NamePropertyId, &.var, &.pCondition
        .pListBox->FindFirst TreeScope_Children, [.pCondition], &.pListItemElement
        cmp [.pListItemElement], 0
        jz .no_item
        .pListItemElement->GetCurrentPattern UIA_SelectionItemPatternId, &.pSelItem
        cmp [.pSelItem], 0
        jz .no_selection_pattern
        .pSelItem->Select
        .pSelItem->Release
.no_selection_pattern:
        .pListItemElement->Release
.no_item:
        .pCondition->Release
        SysFreeString [.var.qVal]

; Cleanup
        .pListBox->Release

... use the afore mentioned inspect.exe to examine your application. Look for UIA_ListControlTypeId to expedite finding the control with a narrower condition.

UI Automation is designed to make most applications accessible.

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

08 Jan 2025, 15:59

Goto page 1, 2 Next

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum