Shell extension problem

Index > Windows > Shell extension problem

Author

Thread

ManOfSteel

Joined: 02 Feb 2005
Posts: 1154

ManOfSteel 04 Aug 2007, 11:35

Hello,

I've been trying for the last few days to make a shell extension that simply adds an item to the context menu and shows the selected file name when the item is clicked.

But now I'm stuck. It just does nothing. I'm surely missing something important.

It also seems there's a reference counting error, for the object is still in use after the server has been unregistered, but I can't find the error.

By the way, I read that the shell calls IContextMenu::QueryContextMenu. I'm wondering how could it possibly know where my implementation of this method is located.

I've attached the code.

Thanks in advance for your help.

Description:		Download
Filename:	dll.asm
Filesize:	20.83 KB
Downloaded:	195 Time(s)

04 Aug 2007, 11:35

SFeLi

Joined: 03 Nov 2004
Posts: 138

SFeLi 05 Aug 2007, 05:00

Sorry, I have no time to look deeper, but here are some errors. IDataObject.GetData fails with E_INVALIDARG because of incorrect FORMATETC structure packing. Correct structure:

Code:

struct                  FORMATETC
cfFormat                dw ?
                        dw ? ;(padding): ptd is dword, so it must be aligned to dword boundary.
ptd                     dd ?
dwAspect                dd ?
lindex                  dd ?
tymed                   dd ?
ends

Incorrect order of methods in IDataObject (not sure):

Code:

interface       IDataObject,\
                QueryInterface,\
                AddRef,\
                Release,\
                GetData,\
                GetDataHere,\
                QueryGetData,\  
                …

You cannot use the same method implementation for different interfaces (“this” needs to be adjusted in the second interface):

Code:

IContextMenuVT          dd QueryInterface  
                        …

If you request IShellExtInit, “this” points to Object+0x00, but if you request IContextMenu, “this” points to Object+0x04, so you need to adjust it, because QueryInterface assumes that it points to Object+0x00.

Code:

IContextMenu_QueryInterface:
   sub  dword[esp+0x04],oSE.lpVtblContextMenu ;Adjust “this”.
   jmp  QueryInterface
;same with AddRef, Release or every other method.

MAKE_HR macro. Please read fasm manual (section 2.3.3):

Code:

   mov     ecx,[severity] 
   shl     ecx,31
   mov     eax,[facility]
   …   
;after macro expansion will become
   mov     ecx,[0]
   shl     ecx,31
   mov     eax,[0]
   …

;
macro MAKE_HR severity,facility,code {
  mov eax,(severity shl 31) or (facility shl 16) or (code)
}

Code:

invoke  MessageBoxEx,[esi + CMINVOKECOMMANDINFO.hwnd],pszFilePath,szContextMenuHandler,MB_OK,0
;must be
invoke  MessageBoxEx,[esi + CMINVOKECOMMANDINFO.hwnd],[pszFilePath],szContextMenuHandler,MB_OK,0

05 Aug 2007, 05:00

ManOfSteel

Joined: 02 Feb 2005
Posts: 1154

ManOfSteel 06 Aug 2007, 12:57

Hello SFeLi.

Thank you for your help.
The extension does its job well now, but the DLL is still in use after unregistering it. When I try to delete the dll, I get a message saying "Cannot delete dll: The specified file is being used by Windows".
I've attached the modified code.

Quote:

IDataObject.GetData fails with E_INVALIDARG because of incorrect FORMATETC structure packing.

Should all Windows structures be aligned to a dword boundary?

Quote:

Incorrect order of methods in IDataObject

I got IDataObject's virtual table from MASM:

Code:

_vtIDataObject MACRO CastName:REQ
    ; IUnknown methods 
    _vtIUnknown CastName
    ; IDataObject methods
    &CastName&_GetData                     comethod3 ?
    &CastName&_RemoteGetData               comethod3 ?
    &CastName&_GetDataHere                 comethod3 ?
    &CastName&_RemoteGetDataHere           comethod3 ?
    &CastName&_QueryGetData                comethod2 ?
    &CastName&_GetCanonicalFormatEtc       comethod3 ?
    &CastName&_SetData                     comethod4 ?
    &CastName&_RemoteSetData               comethod4 ?
    &CastName&_EnumFormatEtc               comethod3 ?
    &CastName&_DAdvise                     comethod5 ?
    &CastName&_DUnadvise                   comethod2 ?
    &CastName&_EnumDAdvise                 comethod2 ?
ENDM

Quote:

You cannot use the same method implementation for different interfaces (?this? needs to be adjusted in the second interface)

I added the following to IContextMenu::AddRef, IContextMenu::Release and IContextMenu::QueryContextMenu.

Code:

mov eax,[this]
sub eax,oSE.lpVtblContextMenu

But are you sure about IContextMenu::QueryInterface, because Explorer crashes when I add the code there. Or am I missing something?

Code:

invoke IsEqualGUID,[pRIID],IID_IContextMenu
.if eax = TRUE
mov eax,[this]
sub eax,oSE.lpVtblContextMenu
push eax
mov eax,[eax]
call [eax + IContextMenu.AddRef]

Quote:

MAKE_HR macro. Please read fasm manual (section 2.3.3):

I could remove the brackets and it would work as it should, but I'll use your solution instead since it's smaller and all the work is done at assembly time.

Description:		Download
Filename:	dll.asm
Filesize:	23.22 KB
Downloaded:	200 Time(s)

06 Aug 2007, 12:57

SFeLi

Joined: 03 Nov 2004
Posts: 138

SFeLi 06 Aug 2007, 15:19

Quote:

Should all Windows structures be aligned to a dword boundary?

Not all, try to search MSSDK header files for “#include <pshpack”.

Quote:

I got IDataObject's virtual table from MASM

Ok, here is IDataObject vtable definition from OBJIDL.H (MSVC6 package):

Code:

    typedef struct IDataObjectVtbl
    {
        BEGIN_INTERFACE
        
        HRESULT ( STDMETHODCALLTYPE __RPC_FAR *QueryInterface )( 
            IDataObject __RPC_FAR * This,
            /* [in] */ REFIID riid,
            /* [iid_is][out] */ void __RPC_FAR *__RPC_FAR *ppvObject);
        
        ULONG ( STDMETHODCALLTYPE __RPC_FAR *AddRef )( 
            IDataObject __RPC_FAR * This);
        
        ULONG ( STDMETHODCALLTYPE __RPC_FAR *Release )( 
            IDataObject __RPC_FAR * This);
        
        /* [local] */ HRESULT ( STDMETHODCALLTYPE __RPC_FAR *GetData )( 
            IDataObject __RPC_FAR * This,
            /* [unique][in] */ FORMATETC __RPC_FAR *pformatetcIn,
            /* [out] */ STGMEDIUM __RPC_FAR *pmedium);
        
        /* [local] */ HRESULT ( STDMETHODCALLTYPE __RPC_FAR *GetDataHere )( 
            IDataObject __RPC_FAR * This,
            /* [unique][in] */ FORMATETC __RPC_FAR *pformatetc,
            /* [out][in] */ STGMEDIUM __RPC_FAR *pmedium);
        
        HRESULT ( STDMETHODCALLTYPE __RPC_FAR *QueryGetData )( 
            IDataObject __RPC_FAR * This,
            /* [unique][in] */ FORMATETC __RPC_FAR *pformatetc);
        
        HRESULT ( STDMETHODCALLTYPE __RPC_FAR *GetCanonicalFormatEtc )( 
            IDataObject __RPC_FAR * This,
            /* [unique][in] */ FORMATETC __RPC_FAR *pformatectIn,
            /* [out] */ FORMATETC __RPC_FAR *pformatetcOut);
        
        /* [local] */ HRESULT ( STDMETHODCALLTYPE __RPC_FAR *SetData )( 
            IDataObject __RPC_FAR * This,
            /* [unique][in] */ FORMATETC __RPC_FAR *pformatetc,
            /* [unique][in] */ STGMEDIUM __RPC_FAR *pmedium,
            /* [in] */ BOOL fRelease);
        
        HRESULT ( STDMETHODCALLTYPE __RPC_FAR *EnumFormatEtc )( 
            IDataObject __RPC_FAR * This,
            /* [in] */ DWORD dwDirection,
            /* [out] */ IEnumFORMATETC __RPC_FAR *__RPC_FAR *ppenumFormatEtc);
        
        HRESULT ( STDMETHODCALLTYPE __RPC_FAR *DAdvise )( 
            IDataObject __RPC_FAR * This,
            /* [in] */ FORMATETC __RPC_FAR *pformatetc,
            /* [in] */ DWORD advf,
            /* [unique][in] */ IAdviseSink __RPC_FAR *pAdvSink,
            /* [out] */ DWORD __RPC_FAR *pdwConnection);
        
        HRESULT ( STDMETHODCALLTYPE __RPC_FAR *DUnadvise )( 
            IDataObject __RPC_FAR * This,
            /* [in] */ DWORD dwConnection);
        
        HRESULT ( STDMETHODCALLTYPE __RPC_FAR *EnumDAdvise )( 
            IDataObject __RPC_FAR * This,
            /* [out] */ IEnumSTATDATA __RPC_FAR *__RPC_FAR *ppenumAdvise);
        
        END_INTERFACE
    } IDataObjectVtbl;

06 Aug 2007, 15:19

ManOfSteel

Joined: 02 Feb 2005
Posts: 1154

ManOfSteel 08 Aug 2007, 15:42

You were right about IDataObject. I checked the MSDN references and there are none of the 'Remote____' methods I mentioned. I really have no idea where MASM32 and colib programmers got these from. I downloaded the latest version (9) and the error is still there.

For the problem with the DLL not getting removed from memory, I noticed it is removed a certain time after unregistration, and the DLL can be deleted after that. I tried to use CoFreeUnusedLibraries but it still remained in use for a while.

Thanks for all your help.

08 Aug 2007, 15:42

SFeLi

Joined: 03 Nov 2004
Posts: 138

SFeLi 08 Aug 2007, 17:08

Quote:

DLL not getting removed from memory

Maybe AlwaysUnloadDll option can help you:

Quote:

The shell automatically unloads a DLL when the DLL's usage count is zero, but only after the DLL has not been used for a period of time. The inactive period may be unacceptably long at times, especially when a shell extension DLL is being debugged. You can shorten the inactive period by adding the following information to the registry.

HKLM
Software
Microsoft
Windows
CurrentVersion
Explorer
AlwaysUnloadDll

AlwaysUnloadDll shortens the inactive period so that DLLs are unloaded quickly.
While debugging your extension, you may want to shut down Windows without closing the currently running applications. To do so, follow these steps:

1. From the Start menu on the Windows taskbar, choose Shut Down.
2. While holding down the CTRL+ALT+SHIFT key combination, click the No button in the Shut Down Windows dialog box.

08 Aug 2007, 17:08

ManOfSteel

Joined: 02 Feb 2005
Posts: 1154

ManOfSteel 09 Aug 2007, 05:47

Quote:

Maybe AlwaysUnloadDll option can help you

I already have AlwaysUnloadDll set up but I've never noticed any difference with or without it.

Quote:

While debugging your extension, you may want to shut down Windows without closing the currently running applications

This technique takes down Explorer with all its instances and all that goes with it (desktop, taskbar, etc).
I've been using a better way: CTRL+ALT+DEL, select the main 'Explorer' process, hit cancel when Windows asks for shutdown, wait a few seconds and finaly terminate Explorer. This takes down the extension more rapidly, while keeping the rest intact (well, almost). And then, Windows is kind enough to run a brand new instance of Explorer.
But all this is still not convenient enough for debugging.

09 Aug 2007, 05:47

Japheth

Joined: 26 Oct 2004
Posts: 151

Japheth 10 Aug 2007, 04:50

ManOfSteel wrote:

But all this is still not convenient enough for debugging.

You can try my explorer clone:

http://www.japheth.de/ExplASM.html

it's compatible with the standard explorer and source is included, so you have both sources, server and client, available. It's written in MASM, though.

10 Aug 2007, 04:50

ManOfSteel

Joined: 02 Feb 2005
Posts: 1154

ManOfSteel 11 Aug 2007, 17:08

Hello Japheth.

I added 'set INCLUDE=C:\MASM32\Win32Inc\Include' to my autoexec.bat but MASM refuses to assemble the source:

Quote:

Assembling: CServiceProvider.asm
Assembling: CShellBrowser.asm
CShellBrowser.asm(837) : error A2006: undefined symbol
Assembling: CDropTarget.asm
Assembling: COleCommandTarget.asm
Assembling: COleInPlaceFrame.asm
Assembling: CVViewer.asm
Assembling: DDEStuff.asm
Assembling: ExplASM.asm
rsrc.rc(10) : fatal error RC1015: cannot open include file 'windows.h'.
LINK : fatal error LNK1181: cannot open input file "CShellBrowser.obj"
LINK : fatal error LNK1181: cannot open input file "CShellBrowser.obj"

And I have no windows.h, neither in Win32Inc nor in MASM32 directories.

11 Aug 2007, 17:08

Japheth

Joined: 26 Oct 2004
Posts: 151

Japheth 12 Aug 2007, 11:03

ManOfSteel wrote:

Hello Japheth.

I added 'set INCLUDE=C:\MASM32\Win32Inc\Include' to my autoexec.bat but MASM refuses to assemble the source:

And I have no windows.h, neither in Win32Inc nor in MASM32 directories.

Sorry!

The rsrc.rc file included windows.h from the platform SDK.
I modified the source so windows.h is no longer needed.

The error flagged by MASM in CShellBrowser.asm has been fixed as well.

12 Aug 2007, 11:03

ManOfSteel

Joined: 02 Feb 2005
Posts: 1154

ManOfSteel 13 Aug 2007, 07:55

You've got some really interesting programs on your site and this explorer clone is a very good one. Good work!

13 Aug 2007, 07:55

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum