flat assembler
Message board for the users of flat assembler.

Index > Windows > Why does dll not work??

Author
Thread Post new topic Reply to topic
AlexP



Joined: 14 Nov 2007
Posts: 561
Location: Out the window. Yes, that one.
AlexP
When I compile my dll and driver program, I am not allowed to call any of the functions from the dll.... Here's the code for the driver:

Code:
format PE GUI 4.0
entry DriverControl
include '%fasminc%\win32a.inc'

section '.code' code readable executable

DriverControl:

        ;End process
        push 0
        call [ExitProcess]

section '.idata' import data readable writeable

  library kernel,'Kernel32.dll',\
          rijndael,'Rijndael.dll'

  import kernel,\
         ExitProcess,'ExitProcess'

  import rijndael,\
         RijndaelGenKey,'RijndaelGenKey',\
         RijndaelEncrypt,'RijndaelEncrypt',\
         RijndaelDecrypt,'RijndaelDecrypt' 
    

And here's the code (exports) for the dll:

Code:
;**************************************
section '.Exports' export data readable
;**************************************

export 'Rijndael.DLL',\
       RD_Encrypt_Block,'RijndaelEncrypt',\
       RD_Decrypt_Block,'RijndaelDecrypt',\
       RD_Expand_Key,'RijndaelGenKey'
                                          
section '.reloc' fixups data discardable
    

Both compile fine, but when I debug it I'm not even importing the dll. What is wrong with my code???
NOTE: The "functions" I export are just labels, not actual 'proc' or anything. Do they need to be?
Post 23 Dec 2007, 20:53
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17662
Location: In your JS exploiting you and your system
revolution
Not enough information. Post all your code.
Post 24 Dec 2007, 00:01
View user's profile Send private message Visit poster's website Reply with quote
gunblade



Joined: 19 Feb 2004
Posts: 209
gunblade
Well, I noticed your Format line doesn't include the word "DLL" at the end, not sure if thats required though. What I'd suggest is checking out the DLL example in the windows fasm release (its under EXAMPLES\DLL). That should show you exactly how to layout a DLL.
Post 24 Dec 2007, 01:13
View user's profile Send private message Reply with quote
AlexP



Joined: 14 Nov 2007
Posts: 561
Location: Out the window. Yes, that one.
AlexP
I based mine off of the example code from FASM, it still does not work. Here's all the code, but I don't think it is of much other help.

Code:
;Rijndael
;Key lengths 128,192,256 bits
;128-bit block cipher

format PE GUI 4.0 DLL
entry RD_Init
include '%fasminc%\win32ax.inc'

;**************************************
section '.Tables' data readable writeable
;**************************************

SBox    RB      256
InvSBox RB      256
t1      RD      256
t2      RD      256
t3      RD      256
t4      RD      256
t5      RD      256
t6      RD      256
t7      RD      256
t8      RD      256
u1      RD      256
u2      RD      256
u3      RD      256
u4      RD      256
RCon    RD      30

;**************************************
section '.Data' data readable writeable
;**************************************
State RD 4
NkT4  DD 0
Nr    DD 0
Nk    DD 0

;**************************************
section '.Cipher' code readable executable
;**************************************

RD_Init:
        push ebp
        mov ebp,esp
        push ebx esi edi
        mov eax,[ebp+0xC] ;fdwReason
        ;DLL procedure switch block
        cmp eax,1
        je DLL_PROCESS_ATTACH_
        cmp eax,2
        je DLL_THREAD_ATTACH_
        cmp eax,3
        je DLL_THREAD_DETACH_
        cmp eax,4
        je DLL_PROCESS_DETACH_
        jmp AfterInit
        DLL_PROCESS_ATTACH_:
                ;*******************************************************************************************
                ;the log and alog tables are used to make the S box. Afterwards they are overwritten by t2
                ;*******************************************************************************************
                alog_table      =t2
                log_table       =t2+256
                GF_modulus      =01bh
                GF_affine       =01fh
                GF_magic        =063h
                ;*******************************************************************************************
                ;make alog and log tables
                ;*******************************************************************************************
                mov     eax,1           ;the first value is 1
                mov     byte[alog_table],al
                mov     byte[log_table],0  ;actually log(0) is not used
                mov     ecx,eax
                .alog:
                ;multiply AL by 3 in GF(2^Cool
                mov     bh,al
                add     bh,bh
                sbb     bl,bl
                and     bl,GF_modulus
                xor     bh,bl
                xor     al,bh
                mov     byte[ecx+alog_table],al
                mov     byte[eax+log_table],cl
                add     cl,1
                jnc     .alog
                mov     byte[eax+log_table],cl     ;set the last value mod 255
                ;*******************************************************************************************
                ;make S box and inverse S box
                ;*******************************************************************************************
                        mov     edx,255
                        mov     ecx,GF_magic
                ;the multiplicative inverse of 0 needs special handling (log and alog can't deal with 0)
                        mov     byte[000+SBox],cl
                        mov     byte[ecx+InvSBox],0
                .sbox:  movzx   eax,byte[edx+log_table]
                        not     al
                        mov     al,byte[eax+alog_table]    ;AL=multiplicative inverse of DL
                        mov     bl,GF_affine
                        mov     cl,GF_magic
                .sbox2: shr     al,1
                        sbb     bh,bh
                        and     bh,bl
                        xor     cl,bh
                        rol     bl,1
                        test    al,al
                        jnz     .sbox2
                        mov     [edx+SBox],cl
                        mov     [ecx+InvSBox],dl
                        sub     edx,1
                        jnz     .sbox
                ;*******************************************************************************************
                ;make t5, t6, t7, t8, u1, u2, u3 & u4
                ;*******************************************************************************************
                        xor     esi,esi
                        xor     edx,edx
                        mov     cl,byte[log_table+0eh]
                        mov     ch,byte[log_table+09h]
                        mov     bl,byte[log_table+0dh]
                        mov     bh,byte[log_table+0bh]
                .ti:    movzx   eax,byte[esi+InvSBox]
                        lea     edi,[eax*4]
                        test    eax,eax
                        movzx   ebp,al
                        jz      .ti2
                        mov     al,byte[eax+log_table]
                        mov     dl,al
                        add     dl,cl
                        adc     dl,0
                        mov     dl,byte[edx+alog_table]
                        shrd    ebp,edx,8
                        mov     dl,al
                        add     dl,ch
                        adc     dl,0
                        mov     dl,byte[edx+alog_table]
                        shrd    ebp,edx,8
                        mov     dl,al
                        add     dl,bl
                        adc     dl,0
                        mov     dl,byte[edx+alog_table]
                        shrd    ebp,edx,8
                        add     al,bh
                        adc     al,0
                        mov     al,byte[eax+alog_table]
                        shrd    ebp,eax,8
                .ti2:   mov     [esi*4+t5],ebp
                        mov     [edi+u1],ebp
                        rol     ebp,8
                        mov     [esi*4+t6],ebp
                        mov     [edi+u2],ebp
                        rol     ebp,8
                        mov     [esi*4+t7],ebp
                        mov     [edi+u3],ebp
                        rol     ebp,8
                        mov     [esi*4+t8],ebp
                        mov     [edi+u4],ebp
                        add     esi,1
                        test    esi,0ffh
                        jnz     .ti
                ;*******************************************************************************************
                ;make t1, t2, t3 & t4
                ;*******************************************************************************************
                        xor     edx,edx
                .t:     movzx   eax,byte[edx+SBox]
                ;the log and alog tables will be destroyed so the multiplications are done directly
                ;multiply AL by 2 in GF(2^Cool
                        mov     ah,al
                        add     ah,ah
                        sbb     bl,bl
                        and     bl,GF_modulus
                        xor     ah,bl   ;ah=al*2
                ;multiply AL by 3 in GF(2^Cool
                        mov     ch,al
                        xor     ch,ah   ;ch=al*3
                ;make the value
                        mov     cl,al
                        shl     ecx,16
                        mov     ch,al
                        mov     cl,ah
                        mov     [edx*4+t1],ecx
                        rol     ecx,8
                        mov     [edx*4+t2],ecx
                        rol     ecx,8
                        mov     [edx*4+t3],ecx
                        rol     ecx,8
                        mov     [edx*4+t4],ecx
                        add     dl,1
                        jnc     .t
                ;******************************************
                ;make rcon
                ;******************************************
                        xor     edx,edx
                        mov     eax,1
                .rcon:  mov     [edx*4+RCon],eax
                ;mulitply AL by 2 in GF(2^Cool
                        add     al,al
                        sbb     bl,bl
                        and     bl,GF_modulus
                        xor     al,bl   ;al=al*2
                        add     dl,1
                        cmp     dl,30
                        jb      .rcon

        jmp AfterInit
        DLL_THREAD_ATTACH_:
                ;Future thread-specifics?
        jmp AfterInit
        DLL_THREAD_DETACH_:
                ;Future thread stuff?
        jmp AfterInit
        DLL_PROCESS_DETACH_:
                ;Nothing here for now...
        AfterInit:
        pop edi esi ebx ebp
        ret 0xC
;***********************************
;     Main Rijndael Cipher
;Parameters:
;    *ExpandedKey
;    *Input
;    *Output
;***********************************
RD_Encrypt_Block:
        push ebp
        mov ebp,esp
        push eax ebx ecx edx edi esi
        mov edx,[ebp+0x8]
        mov esi,[ebp+0xC]
        mov edi,State
        mov ecx,0x4
        rep movs DWORD [edi],DWORD [esi] ;Move In block into State block
        xor esi,esi ;Round
        xor eax,eax
        call RoundKey
        CipherLoop:
        movzx eax,BYTE[State+edi+0]
        movzx ebx,BYTE[State+edi+1]
        movzx ecx,BYTE[State+edi+2]
        mov eax,[eax*4+t1]
        xor eax,[ebx*4+t2]
        movzx ebx,BYTE[State+edi+3]
        xor eax,[ecx*4+t3]
        xor eax,[ebx*4+t4]
        shr edi,2
        lea ecx,[esi*4+edi]
        shl edi,2
        xor eax,[edx+ecx*4] ;ExpandedKey+4*(Round*4+StateOffset)
        mov [State+edi*4],eax
        add edi,4
        cmp edi,16
        jne CipherLoop
        inc esi
        xor edi,edi
        cmp esi,[Nr]
        jne CipherLoop
        ;FinalRound
        mov ecx,[State]
        call SubWord
        mov [State],ecx
        mov ecx,[State+4]
        call SubWord
        rol ecx,8
        mov [State+4],ecx
        mov ecx,[State+8]
        call SubWord
        rol ecx,16
        mov [State+8],ecx
        mov ecx,[State+12]
        call SubWord
        ror ecx,1
        mov [State+12],ecx
        xor eax,eax
        call RoundKey
        ;Ending Sequence
        mov esi,State
        lea edi,[ebp+0x10]
        rep movs DWORD [edi],DWORD [esi]
        EndCipher:
        pop esi edi edx ecx ebx eax ebp
        ret 0xC
;***************************************
;          Rijndael Decryption
;NOTE: For decrypting, the bool param of
;      RijndaelGenKey needs to = TRUE
;Parameters:
;    *ExpandedKey
;    *Input
;    *Output
;***************************************
RD_Decrypt_Block:
        push ebp
        mov ebp,esp
        push eax ebx ecx edx edi esi
        mov edx,[ebp+0x8]
        mov esi,[ebp+0xC]
        mov edi,State
        mov ecx,0x4
        rep movs DWORD [edi],DWORD [esi] ;Move In block into State block
        xor esi,esi ;Round
        xor eax,eax
        call RoundKey
        InvCipherLoop:
        movzx eax,BYTE[State+edi+0]
        movzx ebx,BYTE[State+edi+1]
        movzx ecx,BYTE[State+edi+2]
        mov eax,[eax*4+t5]
        xor eax,[ebx*4+t6]
        movzx ebx,BYTE[State+edi+3]
        xor eax,[ecx*4+t7]
        xor eax,[ebx*4+t8]
        shr edi,2           ;
        lea ecx,[esi*4+edi] ;
        shl edi,2           ;
        xor eax,[edx+ecx*4] ;ExpandedKey+4*(Round*4+StateOffset)
        mov [State+edi*4],eax
        add edi,4
        cmp edi,16
        jne InvCipherLoop
        inc esi
        xor edi,edi
        cmp esi,[Nr]
        jne InvCipherLoop
        ;FinalRound
        mov ecx,[State]
        call SubWordB
        mov [State],ecx
        mov ecx,[State+4]
        call SubWordB
        ror ecx,8
        mov [State+4],ecx
        mov ecx,[State+8]
        call SubWordB
        ror ecx,16
        mov [State+8],ecx
        mov ecx,[State+12]
        call SubWordB
        ror ecx,1
        mov [State+12],ecx
        xor eax,eax
        call RoundKey
        ;Ending Sequence
        mov esi,State
        lea edi,[ebp+0x10]
        rep movs DWORD [edi],DWORD [esi]
        EndInvCipher:
        pop esi edi edx ecx ebx eax ebp
        ret 0xC
;**************************************
;           Key Expansion
;On Stack: *Input Key,*OutputKey,Cipher,bool Decrypt
;**************************************
RD_Expand_Key:
        push ebp
        mov ebp,esp
        push eax ebx ecx edx edi esi
        mov ebx, [ebp+0x8] ;Pointer to input key
        mov edx, [ebp+0xC] ;Pointer to output key
        mov ecx, [ebp+0x10] ;Cipher identifier
        ;Set variables for cipher
        cmp ecx,0x80  ;128 bit key
        jnz Test192
        mov [Nk],4
        mov [Nr],10
        jmp AfterTest
        Test192:
        cmp ecx,0xC0  ;192 bits
        jnz Test256
        mov [Nk],6
        mov [Nr],12
        jmp AfterTest
        Test256:
        cmp ecx,0x100 ;256 bits
        jnz Enda
        mov [Nk],8
        mov [Nr],14
        AfterTest:
        ;Place input into first Nk bytes of expanded key
        mov ecx,[Nk]
        mov edi,edx
        mov esi,ebx
        rep movs DWORD [edi],DWORD [esi]
        xor edx,edx
        mov eax,[Nk]
        shl eax,2
        mov [NkT4],eax
        lea ebx,[Nr+1*4]
        lea ebx,[ebx*4]
        BeginWhile:
        ;Last key entry -> ecx
        sub eax,4
        mov ecx,[edx+eax]
        add eax,4
        push edx
        push eax
        div [NkT4]
        mov edi,edx
        mov esi,eax
        pop eax
        pop edx
        cmp edi,0
        jne ElseIf
        rol ecx,8
        call SubWord
        ElseIf:
        cmp edi,16
        jne EndIf
        cmp [Nk],8
        jne EndIf
        call SubWord
        EndIf:
        sub eax,[NkT4]
        mov eax,[edx+eax]
        xor eax,ecx
        mov [edx+eax],eax
        add eax,4
        cmp eax,ebx
        jne BeginWhile
        cmp dword [ebp+0x14],0
        je Enda
        ;Make decryption round keys instead
        ;Apply u-table xor'ing to each key except first and last
        ;esi is the counter (by ones)
        ;edi is the byte offset for current word of key schedule
        ;Skip the first and last round keys
        ;A round key is static 16 bytes!!! (4 words)
        ;That's because there's 4 columns in state, Nb for AES
        mov esi,1
        mov edi,edx
        add edi,16 ;Skip first round key
        StartDecryptKeys:
        movzx eax,byte[edi]
        movzx ebx,byte[edi+1]
        movzx ecx,byte[edi+2]
        movzx edx,byte[edi+3]
        mov eax,[eax*4+u1]
        xor eax,[ebx*4+u2]
        xor eax,[ecx*4+u3]
        xor eax,[edx*4+u4]
        mov [edi],eax
        add edi,16
        inc esi
        cmp esi,[Nr]  ;Skip last round key
        jne StartDecryptKeys
        Enda:
        pop esi edi edx ecx ebx eax ebp
        ret 0xC
;*************************************
;          Helper Functions
;      SubWord,SubWordB,RoundKey
;*************************************
SubWord:
        push eax ebx edx ecx
        movzx eax,byte[esp]
        movzx ebx,byte[esp+1]
        movzx ecx,byte[esp+2]
        movzx edx,byte[esp+3]
        mov al,byte[SBox+eax]
        mov ah,byte[SBox+ebx]
        mov bl,byte[SBox+ecx]
        mov bh,byte[SBox+edx]
        movzx ecx,ax
        shl ecx,16
        or ecx,ebx
        add esp,0x4
        pop edx ebx eax
        ret
SubWordB:
        push eax ebx edx ecx
        movzx eax,byte[esp]
        movzx ebx,byte[esp+1]
        movzx ecx,byte[esp+2]
        movzx edx,byte[esp+3]
        mov al,byte[InvSBox+eax]
        mov ah,byte[InvSBox+ebx]
        mov bl,byte[InvSBox+ecx]
        mov bh,byte[InvSBox+edx]
        movzx ecx,ax
        shl ecx,16
        or ecx,ebx
        add esp,0x4
        pop edx ebx eax
        ret
RoundKey:
        lea ebx,[esi*4+eax]  ;Round*4 + State word count
        mov ecx,[eax*4+State]
        mov ebx,[edx+ebx]     ;Key Schedule word
        xor ebx,ecx
        mov [eax*4+State],ebx ;Store into state again
        inc eax
        cmp eax,4
        jne RoundKey
        ret
;**************************************
section '.Exports' export data readable
;**************************************

export 'Rijndael.DLL',\
       RD_Encrypt_Block,'RijndaelEncrypt',\
       RD_Decrypt_Block,'RijndaelDecrypt',\
       RD_Expand_Key,'RijndaelGenKey'
                                          
section '.reloc' fixups data discardable
    

Here's the "driver" program I made
Code:
format PE GUI 4.0
entry DriverControl
include '%fasminc%\win32a.inc'

section '.data' data readable writeable
hStdOut  DD 0
hStdIn   DD 0
ReadChar RB 20
NumRead  RB 2
NumWritten RB 2
MainScreen DB 'Rijndael Driver',0x0D,0x0A,0

;Cipher key has Nk words (dwords)
CipherKeyWord0   DD 0x603deb10
CipherKeyWord1   DD 0x15ca71be
CipherKeyWord2   DD 0x2b73aef0
CipherKeyWord3   DD 0x857d7781
CipherKeyWord4   DD 0x1f352c07
CipherKeyWord5   DD 0x3b6108d7
CipherKeyWord6   DD 0x2d9810a3
CipherKeyWord7   DD 0x0914dff4
;Here's for the expanded key
ExpandedKey      RD 60

section '.code' code readable executable

DriverControl:

        push 0
        push 0x100
        push ExpandedKey
        push CipherKeyWord0
        call RijndaelGenKey

        ;End process
        push 0
        call [ExitProcess]



section '.idata' import data readable writeable

  library kernel,'Kernel32.dll',\
          rijndael,'Rijndael.dll'

  import kernel,\
         ExitProcess,'ExitProcess'

  import rijndael,\
         RijndaelGenKey,'RijndaelGenKey',\
         RijndaelEncrypt,'RijndaelEncrypt',\
         RijndaelDecrypt,'RijndaelDecrypt'
    

When I compile now, it is working?! I try following the call instruction of the driver to the dll, but it goes to some non-sense code of repeat instructions... Please tell me what happened! or try to compile it yourself to see how crazy it is?!

EDIT: Okay, I guess it was my computer. I shut it off, then on again and the above happened. I recognized that I didn't use the [] brackets, so I put them in and the dll is working now. Now, I must see if the dll initialization code is working, Is the address spaces the same in OllyDbg? Does anyone know how to test this? maybe create a messagebox or something?


Last edited by AlexP on 24 Dec 2007, 01:38; edited 1 time in total
Post 24 Dec 2007, 01:29
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17662
Location: In your JS exploiting you and your system
revolution
Try this:
Code:
        call [RijndaelGenKey]    
Post 24 Dec 2007, 01:35
View user's profile Send private message Visit poster's website Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
You correctly used square brackets for the call to ExitProcess, now do the same for your DLLs, otherwise you are calling the import table entry instead of the actual DLL function.
Post 24 Dec 2007, 01:37
View user's profile Send private message Reply with quote
AlexP



Joined: 14 Nov 2007
Posts: 561
Location: Out the window. Yes, that one.
AlexP
sry, I've got an even bigger problem now. Please check it out in the "Need higher opinion of function" thread, pg. 2. The dll is being corrupted or something on runtime.
Post 24 Dec 2007, 01:46
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on GitHub, YouTube, Twitter.

Website powered by rwasa.