flat assembler
Message board for the users of flat assembler.

Index > DOS > Disassembler Instructions VS. Hex'ed Instructions?

Author
Thread Post new topic Reply to topic
rytz



Joined: 24 Dec 2013
Posts: 2
rytz 25 Dec 2013, 01:59
Hi:

I'm trying to understand the difference between instruction displayed by disassemblers and the actual instruction shown in a DLL file (through a hex editor).

----------
DLL File(s): 16-bit MS-DOS DLL (protected mode)
----------

Scenario: Version 1 of this DLL works fine, but version 2 is crashing on a database insert (BTRIEVE). The main difference appears to be the size of the data segment that is referenced (trying to insert too much data).

When comparing disassembled versions 1 and 2 of the DLL, it appears that in the non-working version (2), the wrong data segment is being referenced and is about 4 times larger than in version 1.

In both versions of the DLL, the file hex instruction is shown as "B8 FF FF". My understanding is that this instruction is moving immediate value 65535 to register AX. However, I don't fully understand how the disassembler determines what data segment is actually being chosen.

------
Disassembled DLL: Version 1 (working)
------
Assembly:
Code:
mov     ax, seg dseg32    

Hex:
Code:
B8 EB 7B    


------
Disassembled DLL: Version 2 (non-working)
------
Assembly:
Code:
mov     ax, seg dseg66    

Hex:
Code:
B8 37 AB    


------
DLL Hex Editor Instructions (both versions)
------
Hex:
Code:
B8 FF FF    


As shown above, the disassembler shows different data segments used for both versions, but in the file through a hex editor, both versions just show "FF FF".

Question: How can I change the data segment that is being pointed to if the actual hex instruction is always "FF FF"? If the data segment chosen is determined at runtime, how would I go about doing some type of redirection? How does the program determine which segment to use?

The goal would be to try and force the program to use a different data segment in the MOV call. At least to test my theory out.

Using the above info, if data segment "35" appears to be the correct data segment, how can I tell the DLL to use segment 35 instead of 66?

Is it possible to do this?

Thanks.
Post 25 Dec 2013, 01:59
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20405
Location: In your JS exploiting you and your system
revolution 25 Dec 2013, 02:07
This is probably a relocation place holder. The value 0xFFFF has no meaning and the actual value is placed by the OS loader at program start-up.

But if this is something else (we don't know since you never gave us any code) then you would need to look deeper and see what is really going on with a debugger.
Post 25 Dec 2013, 02:07
View user's profile Send private message Visit poster's website Reply with quote
rytz



Joined: 24 Dec 2013
Posts: 2
rytz 25 Dec 2013, 02:28
revolution wrote:
This is probably a relocation place holder. The value 0xFFFF has no meaning and the actual value is placed by the OS loader at program start-up.

But if this is something else (we don't know since you never gave us any code) then you would need to look deeper and see what is really going on with a debugger.

Thanks for the reply.

I'm using IDA for disassembling but I'm having problems getting access to the IDA forums or I would have posted this there.

This is an older program that I am trying to run and don't have access to any source code. I feel thankful just to at least have a working version of the DLL (older version 1). The newer version of the DLL provides some features that I would like to have though.

I can paste a bigger snippet of the disassembled code but the DLL itself is almost a 1MB so the complete disassembled code is lengthy.

V1 (working):
Code:
cseg02:172E                 public _SAVE
cseg02:172E _SAVE    proc far
cseg02:172E
cseg02:172E
cseg02:172E var_4           = dword ptr -4
cseg02:172E arg_0           = word ptr  6
cseg02:172E
cseg02:172E                 enter   4, 0
cseg02:1732                 push    si
cseg02:1733                 push    di
cseg02:1734                 push    ds
cseg02:1735                 mov     ax, seg dseg32
cseg02:1738                 mov     ds, ax
cseg02:173A                 assume ds:dseg32
cseg02:173A                 push    [bp+arg_0]    


V2 (non-working):
Code:
cseg25:35A8                 public _SAVE
cseg25:35A8 _SAVE    proc far
cseg25:35A8
cseg25:35A8
cseg25:35A8 var_4           = dword ptr -4
cseg25:35A8 arg_0           = word ptr  6
cseg25:35A8
cseg25:35A8                 enter   4, 0
cseg25:35AC                 push    si
cseg25:35AD                 push    di
cseg25:35AE                 push    ds
cseg25:35AF                 mov     ax, seg dseg66
cseg25:35B2                 mov     ds, ax
cseg25:35B4                 assume ds:dseg66
cseg25:35B4                 push    [bp+arg_0]    


Since this is a DOS-Based 16-bit application, I've been having to run this program under DOSBox. This program also uses the PharLap extender. The only debugger that I've gotten to execute the parent EXE file is DEBUGX. DOSBox's own debugger and GRDB both crash or freeze the program. I'm still having a hard time getting a breakpoint to even stick, though.

If this is some type of relocation, any other suggestions on how to attempt changing the data that is used for B8?
Post 25 Dec 2013, 02:28
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20405
Location: In your JS exploiting you and your system
revolution 25 Dec 2013, 02:37
That code you posted doesn't help us in any way.

If you are 100% certain that the problem is that the wrong segment is being referenced then you can alter the DLL file's relocation table to make it any segment you want.

I suspect the actual problem is elsewhere from what you think it is. It would be very unlikely that an incorrect segment is being used in this type of situation.
Post 25 Dec 2013, 02:37
View user's profile Send private message Visit poster's website Reply with quote
baldr



Joined: 19 Mar 2008
Posts: 1651
baldr 25 Dec 2013, 07:32
rytz,

You may host relevant files somewhere and post link.
Post 25 Dec 2013, 07:32
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.