Message board for the users of flat assembler.
> Blog > Learning binary file formats (work in progress)
This is an early stage of my upcoming book-like tutorial on the executable/object formats, with a heavy assistance of fasmg. I start with PE and I plan to follow up with ELF, then perhaps Mach-O and possibly some others.
You may follow me on Twitter for a little sidenotes and outtakes that I post while writing the main text.
Getting started with fasmg
PE (Portable Executable)
1.1 Building a simple program
1.2 Adding relocations
1.3 Making a library
1.4 Embedding resources
Last edited by Tomasz Grysztar on 16 Aug 2018, 08:57; edited 30 times in total
|15 Jul 2018, 11:48||
Getting started with fasmg
To learn the inner workings of binary files we are going to construct some examples of them manually, with help of fasmg. This is a command line tool that takes the source text, which is like a script that defines how to assemble the binary file out of its components (down to bytes or even individual bits) and saves such produced file under a given name:
fasmg source.asm output.bin
The source contains a series commands, each on its own line of text. One of the basic instructions is DB, which defines data as a series of bytes:
If the source text looks like above, fasmg is going to produce a file that contains three bytes with the given values (that happen to be the ASCII codes of digits from 1 to 3).
The definitions of data can use units larger than a byte, among other available instruction there is DW to define 16-bit (2-byte) "words", DD for 32-bit (4-byte) "double words" and DQ for 64-bit (8-byte) "quad words". They all store values as little-endian (there are easy methods to define big-endian data too, but we are not going to need them here).
A data can also be defined as a string of bytes copied as-is from the source text. Such sequence of characters needs to be enclosed with either single or double quotes:
The DUP operator allows to define many duplicates of the same value:
db 3 dup '!'
Any definition of data has assigned an address, starting from zero. Data can be given a name, by writing it before the DB or other similar instruction as a so-called label:
This name can then be used in expressions and its value is the address of the first byte of the data that it labeled. The following produces a 32-bit value equal to the address of "digits" (most likely zero):
digits db 49,50,51,52,53,54,55,56,57
What makes the assembler especially useful is that we can define various things out of order and fasmg is going to compute and put the right values in the right places, like:
The file generated from the above sample is going to start with a 32-bit value equal to the difference between the "null" and "digits" addresses, that is the length of the string of digits.
dd null - digits digits db 49,50,51,52,53,54,55,56,57 null db 0
A label can also be created without a data definition on the same line, in such case the name needs to be followed by a colon:
A name can also be assigned any computed value with the = or := operator. The := defines a constant, like a label, while = defines a variable whose value may be changed by another similar assignment later.
dd eof db 'Hello!' eof:
The $ is a special name that always equals to the current address:
dd length digits db 49,50,51,52,53,54,55,56,57 null db 0 length := null - digits
Assigning the value of $ to a name has the same effect as defining such named label.
dd length digits db 49,50,51,52,53,54,55,56,57 length := $ - digits db 0
Various portions of executable files may end up loaded to a different addresses in memory. The instruction ORG allows to change the assumed address for the data definitions that follow, without altering the position in file. This changes the value of $ and the values of all labels defined after this point. Since this decouples $ from the position within the generated file, there is another special name $% that always equals to the position in file regardless of the assumed address.
The above sample contains some comments, in assembly language they are started with a semicolon.
org 0x100 start: offset = $% dd start ; equals 0x100 (256) dd offset ; equals 0
A data can be defined with ? in place of its value. This creates a so-called reserved bytes, which are stripped when at the end of file but not when they are followed by a regular data.
When a reserved byte ends up in the output file, it has a zero value.
db ? ; this one is not cut db 'a' db ? ; this one is cut
The SECTION instruction in the language of fasmg is very similar to ORG, except that it strips all the reserved bytes that were defined just before it, similarly to how they are normally stripped at the end of file. This is going to become useful when the file we make needs to contain sections that are larger in memory than in the file, with additional bytes reserved at the end of each section.
In addition to special symbols $ and $% there is also $%%. It equals the current offset in file and unlike $% it does not count the reserved bytes (which may end up discarded):
To get a better view of the things that get assembled we can use the attached "listing.inc" script to generate a listing of generated bytes next to the instructions that made them. To use this extension we can put the following command as the first line of our source:
dd SIZE_IN_FILE dd SIZE_IN_MEMORY section 0x1000 ADDRESS_IN_MEMORY = $ OFFSET_IN_FILE = $% db 'example' db 0x30 dup ? SIZE_IN_MEMORY := $ - ADDRESS_IN_MEMORY SIZE_IN_FILE := $%% - OFFSET_IN_FILE
Alternatively, we can insert this line into assembly with the -i switch from the command line. Depending on the operating system we may need to escape the space character in different ways. In Windows this may look like:
and in Linux:
fasmg source.asm output.bin -i "include 'listing.inc'"
The listing should be saved in file "output.lst" and look like this:
fasmg source.asm output.bin -i include\ 'listing.inc'
The value in square brackets is the assumed address in memory at which the instruction is assembled, the number with a colon is the offset within the file at which the bytes are written and what follows are their values (everything in hexadecimal).
 00000000: 07 00 00 00 dd SIZE_IN_FILE  00000004: 37 00 00 00 dd SIZE_IN_MEMORY  section 0x1000  ADDRESS_IN_MEMORY = $  OFFSET_IN_FILE = $%  00000008: 65 78 61 6D 70 6C 65 db 'example'  db 0x30 dup ?  SIZE_IN_MEMORY := $ - ADDRESS_IN_MEMORY  SIZE_IN_FILE := $%% - OFFSET_IN_FILE
When the assembly scripts we write become more complex, we may notice some repeating patterns of commands that would become much more pleasant if we could replace them with specialized instructions. To aid with this, the assembler allows us to define macro-instructions. This way we can define a new command, named however we wish, and make it execute a customized sequence of instructions every time it is encountered. For example:
defines a new instruction that can be used like:
macro distanceto address dd address - $ end macro
and it then simply executes the following command:
Naturally, macro-instructions often end up being much more complex. Let us have a look at one that is almost as simple as the above one, but is going to be very useful to us on numerous occasions:
dd digits - $
The ? after the name of a macro makes the name case-insensitive. The first of the arguments is marked with * to tell the assembler that it is required, while the second argument is optional with default value being a question mark. So if we use the macro like this:
macro align? pow2*,value:? db (-$) and (pow2-1) dup value end macro
the instruction that gets executed is:
This macro allows to align the structures we generate so that the address of the next data we define is a multiple of a given power of two.
db (-$) and (4-1) dup ?
We are going to learn more about the language of fasmg when a need arises, but at the moment we know enough to start making some files.
Last edited by Tomasz Grysztar on 13 Aug 2018, 17:41; edited 4 times in total
|02 Aug 2018, 02:22||
PE (Portable Executable)
The road we are going to take is to learn the inner workings of file formats by constructing some files from scratch. This approach is focused on experimentation and the samples will be easy to modify so that one can play with them and perhaps learn even more this way.
The first file we construct is going to be an executable for Windows operating system, in the format called Portable Executable.
PE was designed in 1993 for Windows NT (the first 32-bit system in the family), and has been used from then on by the 32-bit and 64-bit implementations of Windows. Subsequently it has been adopted for some other uses, like EFI, but at this time we are going to focus on its original environment.
1.1 Building a simple program
Before we go on, a few preparations. We should take the ALIGN macro we discussed earlier, it is going to become useful quite soon. We may also need to create some machine code for the actual program inside our executable and for this we need to include an instruction set for a processor architecture we need to work with. Our first choice is going to be very traditional, the 32-bit x86 architecture, so we include an instruction set for processors compatible with 80386:
USE32 is a command provided by the '80386.inc' package, it chooses to assemble instructions for 32-bit mode (if we did not specify it, the default mode would be 16-bit, for historical reasons).
include '80386.inc' use32
A term that often pops up when discussing PE files is the program image. This refers to the layout of the program after it is loaded into memory to be executed, which is not necessarily the same as the structure of the program in the file on disk. The executable needs to define a mapping of sections of the file onto the corresponding areas in memory.
Nevertheless, both the file on disk and image in memory start the same way - with the headers. These structures from the beginning of file become the initial portion of loaded program, at the address called the base of the image. All the other sections created in memory have to be placed after that.
Any PE executable is constructed with an assumed value for the base of the image, for 32-bit programs this is usually 0x400000. We are going to define a constant with this value and use it as the base for our labels:
Therefore all the labels that we define are going to correspond to addresses in the program image.
IMAGE_BASE := 0x400000 org IMAGE_BASE
The next two constants choose the alignment settings for the disk and for the memory. This is one of the sources of discrepancy between the layouts of the file and of the image.
The standard choice of file alignment makes sure that every section in the file starts on a new sector of the disk (traditionally hard drives have a sector size of 512 bytes), to optimize the performance of reading and mapping a single section into memory.
FILE_ALIGNMENT := 512 SECTION_ALIGNMENT := 4096
In memory, the sections are aligned to the size of page (which is 4096 bytes in the basic setup of x86 CPU). This is partly because memory can be allocated only in such increments, but also because different sections may require distinct attributes for the memory (like write-protection) and CPU can have them set up only for entire pages at once.
These constants are better left with the standard values. While it is possible to tweak them in such way that it should still be possible for the operating system to construct the image, the loader may distrust and refuse to load an executable with a non-standard layout. There are also some additional constraints if chosen alignment in memory is smaller than the size of page (we may get back to it later).
It is time to start writing the headers. The very first bytes of the file are usually an unique signature of the format, but in the case of PE a matter is a bit more complicated. At the time when PE format was designed DOS was still a popular operating system and many of the new formats - like NE (16-bit format used by the earliest versions of Windows), LE (used by OS/2, but also by drivers in Windows 9x) and finally PE - were based on the old MZ format used for the .EXE files in DOS. All these formats were made in such way that the initial portion of the file was a valid MZ program that could be executed by DOS, usually it was a tiny program that just displayed a message like "This program cannot be run in DOS mode". This small program was called a stub and its MZ header was extended to contain a special field, ignored by older software, containing the offset of the actual new executable header later in the file.
This way it was even possible to have an executable that would contain two versions of the same software - one for DOS and one for Windows. This was not an usual thing to do, though. Mostly, the stub programs were just informing in one way or the other that the file was not intended to be run from DOS.
Nowadays we do not need to worry much about someone mistakenly trying to execute our PE file in DOS, therefore we are going to make a minimal stub - not a real program, just something that resembles one enough for our PE executable to be valid:
What is important here is that at the position 0x3C from the beginning of MZ header there should be a 32-bit field containing the offset to actual PE header. We fill most of the MZ header with zeros up to that point, normally there are some fields important for the MZ format, but we do not intend to make a functional DOS program.
Stub: .Signature dw "MZ" .BytesInLastSector dw SIZE_OF_STUB mod 512 .NumberOfSectors dw (SIZE_OF_STUB-1)/512 + 1 .NumberOfRelocations dw 0 .NumberOfHeaderParagraphs dw SIZE_OF_STUB_HEADER / 16 db 0x3C - ($-Stub) dup 0 .NewHeaderOffset dd Header-IMAGE_BASE align 16 SIZE_OF_STUB_HEADER := $ - Stub ; The code of a DOS program would go here. SIZE_OF_STUB := $ - Stub
We compute the offset of a main PE header by subtracting IMAGE_BASE from its address (available through a label that we are going to define below). For all the headers there is such clear correspondence between addresses in image and positions in file.
We also fill a couple of fields in the MZ header that are crucial for its integrity, namely the size of the header and of the entire program. The header is measured in 16-byte units (in DOS they were called paragraphs) and the "align 16" is there to make sure that this is a multiple of 16 (though in this case nothing needs to be done, the position immediately after the NewHeaderOffset is 64). The size of DOS program is given as a count of 512-byte sectors, but the last one of them is allowed to be not fully filled and BytesInLastSector gives the number of bytes in it.
On a side note, when a label starts with a dot, it belongs to the namespace of a regular label that preceded it. The labels defined here could be accessed from elsewhere with identifiers like "Stub.Signature" or "Stub.NewHeaderOffset".
With the stub ready, we can move on to the main header, this is where the actual PE signature is going to be. This header must be aligned on 8-byte boundary, hence we put an "align 8" here, though it again does nothing (but if we had put a real DOS program above, the position in file might have been misaligned).
There are some constants used here that are given names in the official specifications of PE format. To make the generated data more tangible in the first demonstration, we use their values directly and leave the names in the comments. But as we continue to work with these examples, later we may prefer to include an additional header into our script with the definitions of all these constants and just use the names.
align 8 Header: .Signature dw "PE",0 .Machine dw 0x14C ; IMAGE_FILE_MACHINE_I386 .NumberOfSections dw NUMBER_OF_SECTIONS .TimeDateStamp dd %t .PointerToSymbolTable dd 0 .NumberOfSymbols dd 0 .SizeOfOptionalHeader dw SectionTable - OptionalHeader .Characteristics dw 0x102 ; IMAGE_FILE_32BIT_MACHINE + IMAGE_FILE_EXECUTABLE_IMAGE
According to the plan, the first example is going to be for a 32-bit mode of a x86 CPU and we state this in the Machine field, but also by including IMAGE_FILE_32BIT_MACHINE value in the Characteristics. The latter field is a set of flags and there is another one that we unquestionably need there - IMAGE_FILE_EXECUTABLE_IMAGE tells that the file contains an executable code.
PE is closely related to COFF, which is a format of object files that are created by compilers as an intermediate stage before they are finally linked to create code that can be executed. These two formats have mostly identical headers (except for the PE signature, which is missing in COFF) and they share the values of various constants. The value of IMAGE_FILE_EXECUTABLE_IMAGE has been used by COFF to distinguish the object files from the executable ones (when we later talk about ELF format, which superseded COFF on the Unix systems, we are going to see that it has similar variants).
In NumberOfSections we need to state how many sections do we plan to create. We do not know that yet, but we can use the name of a constant that we define later with the right value.
TimeDateStamp needs to tell when the file was created, in the "milliseconds since Unix epoch" format. A special symbol %t is provided by fasmg with such value.
PointerToSymbolTable and NumberOfSymbols are another relic of the COFF format. They are not used in PE and we just fill them with zeros.
After the main header comes the so-called "optional header". This name is also a legacy of COFF, as this structure contains a crucial information about the entry point of an executable code and is definitely required for any PE image. It was only optional in COFF, when the file could be an intermediate object, not yet made into an executable.
The optional header follows immediately after the main one and is in turn followed by the section table. Thus to obtain the size that we need to put in SizeOfOptionalHeader we just compute the difference between the OptionalHeader and SectionTable addresses.
OptionalHeader: .Magic dw 0x10B .MajorLinkerVersion db 0 .MinorLinkerVersion db 0 .SizeOfCode dd 0 .SizeOfInitializedData dd 0 .SizeOfUninitializedData dd 0 .AddressOfEntryPoint dd EntryPoint-IMAGE_BASE .BaseOfCode dd 0 .BaseOfData dd 0
The value of Magic identifies a variant of PE format. For classic 32-bit PE it is always 0x10B (a ZMAGIC value which COFF inherited from the old a.out format); while 0x20B is used to mark PE+ files, a variety intended mainly for 64-bit architectures. They slightly differ in format of the structures that follow, we are going to look at these differences later, when we create a 64-bit executable.
Of the other fields in this initial portion of the "optional" header the only important one is AddressOfEntryPoint, which should contain an address of entry point relative to the base of the image. The specification calls this kind of value an RVA (Relative Virtual Address), while VA (Virtual Address) is just a direct address in memory. To compute an RVA we simply subtract IMAGE_BASE from the address (VA). The EntryPoint label is going to be defined later, in the code of our program.
MajorLinkerVersion and MinorLinkerVersion are filled by a linker when it creates the executable, this allows the linker to put some mark of authorship on the executable. We are not a linker, so we can decide for ourselves what kind of mark to leave there. A simple choice is just zeros.
The other fields, like SizeOfCode and AddressOfCode, are remnants of the original COFF model (which in turn inherited them from the old a.out) and they do not really matter to PE loader. Various kinds of code and data sections may be intermixed within the image and the true authority on their sizes and placement is held by the section table. The fields here are just a supplementary information and, for instance, if there were several sections of data with some code in-between, the sum of their sizes would serve only a statistical role.
If we wanted to be pedantic about it, we could fill these fields with values copied from our section table, but for now we just leave them zeroed. An additional sign of the irrelevancy of these numbers is that in PE+ the entire BaseOfData field is readily sacrificed to allow the subsequent ImageBase field to be enlarged to 64-bit without moving the later ones.
.ImageBase dd IMAGE_BASE .SectionAlignment dd SECTION_ALIGNMENT .FileAlignment dd FILE_ALIGNMENT .MajorOperatingSystemVersion dw 3 .MinorOperatingSystemVersion dw 10 .MajorImageVersion dw 0 .MinorImageVersion dw 0 .MajorSubsystemVersion dw 3 .MinorSubsystemVersion dw 10 .Win32VersionValue dd 0 .SizeOfImage dd SIZE_OF_IMAGE .SizeOfHeaders dd SIZE_OF_HEADERS .CheckSum dd 0 .Subsystem dw 2 ; IMAGE_SUBSYSTEM_WINDOWS_GUI .DllCharacteristics dw 0 .SizeOfStackReserve dd 4096 .SizeOfStackCommit dd 4096 .SizeOfHeapReserve dd 65536 .SizeOfHeapCommit dd 0 .LoaderFlags dd 0 .NumberOfRvaAndSizes dd NUMBER_OF_RVA_AND_SIZES
In contrast, this part of headers holds many important values. All the constants we defined earlier - the base of the image and the alignment values - are stored here exactly as they are. We also use two constants we have not yet defined to fill SizeOfImage and SizeOfHeaders, we are going to calculate these values later.
MajorOperatingSystemVersion together with MinorOperatingSystemVersion as well as MajorSubsystemVersion with MinorSubsystemVersion declare what version of operating system is needed to execute the image. Programs created for older versions are allowed to run on the newer ones, and this example is not going to use any features that were not in Windows since the beginning, so to not unnecessarily limit the execution of program we put 3.10 there (this is the version number of first Windows NT that supported PE format).
MajorImageVersion and MinorImageVersion could indicate the version of our program, but they are usually unused. And Win32VersionValue is just a reserved field, with currently unknown purpose; it needs to be zero. The same goes for LoaderFlags further below.
CheckSum is a value computed over all the bytes of the executable that can be used to check whether the file has been modified in any way since the time when it was calculated. Normal programs are not required to have a valid checksum, so in this example we are going to skip this step. But even when we plan to compute the checksum, the value of this field should not partake in the summation so it is better to have it initially zeroed.
Subsystem identifies the environment where the program wants to be run. For normal applications this is either GUI or console.
DllCharacteristics is an additional set of flags supplementary to Characteristics in the main header. This is another case of a misnomer, the flags here are not necessarily related to whether the file is a DLL. Nevertheless, at the moment we do not need to set any of them.
SizeOfStackReserve and SizeOfStackCommit set up the size of stack for our program, the former states how large the stack is allowed to become, while the latter determines the initial size. We go with a single page for both. SizeOfHeapReserve and SizeOfHeapCommit provide similar settings for the local heap, which is a pool from which program may allocate small blocks of memory whenever needed. We set up some usual values, though we are not going to use heap in our simple program.
Finally, NumberOfRvaAndSizes specifies how many pairs consisting of a relative address and a size follow immediately after. This forms a sort of catalogue of specialized data structures present in the image. They come in a fixed order, as folows:
Out of many possible tables that PE image may declare this way, we provide just one. The import table is necessary for us to gain access to the functions of Windows API. When we define it below, we need to demark it with ImportTable and ImportTable.End labels.
RvaAndSizes: .Export.Rva dd 0 .Export.Size dd 0 .Import.Rva dd ImportTable-IMAGE_BASE .Import.Size dd ImportTable.End-ImportTable .Resource.Rva dd 0 .Resource.Size dd 0 .Exception.Rva dd 0 .Exception.Size dd 0 .Certificate.Rva dd 0 .Certificate.Size dd 0 .BaseRelocation.Rva dd 0 .BaseRelocation.Size dd 0 .Debug.Rva dd 0 .Debug.Size dd 0 .Architecture.Rva dd 0 .Architecture.Size dd 0 .GlobalPtr.Rva dd 0 .GlobalPtr.Size dd 0 .TLS.Rva dd 0 .TLS.Size dd 0 .LoadConfig.Rva dd 0 .LoadConfig.Size dd 0 .BoundImport.Rva dd 0 .BoundImport.Size dd 0 .IAT.Rva dd 0 .IAT.Size dd 0 .DelayImport.Rva dd 0 .DelayImport.Size dd 0 .COMPlus.Rva dd 0 .COMPlus.Size dd 0 .Reserved.Rva dd 0 .Reserved.Size dd 0
Here the optional header ends, immediately followed by the section table - a crucial component of the headers.
Our table contains two records, defining two sections with different attributes. The '.text' is a usual name for a section containing executable code, in other words: the text of the program. The '.rdata' is going to contain all kinds of read-only data we need, this should be enough for the first sample.
SectionTable: .1.Name dq +'.text' .1.VirtualSize dd Section.1.End - Section.1 .1.VirtualAddress dd Section.1 - IMAGE_BASE .1.SizeOfRawData dd Section.1.SIZE_IN_FILE .1.PointerToRawData dd Section.1.OFFSET_IN_FILE .1.PointerToRelocations dd 0 .1.PointerToLineNumbers dd 0 .1.NumberOfRelocations dw 0 .1.NumberOfLineNumbers dw 0 .1.Characteristics dd 0x60000000 ; IMAGE_SCN_MEM_EXECUTE + IMAGE_SCN_MEM_READ .2.Name dq +'.rdata' .2.VirtualSize dd Section.2.End - Section.2 .2.VirtualAddress dd Section.2 - IMAGE_BASE .2.SizeOfRawData dd Section.2.SIZE_IN_FILE .2.PointerToRawData dd Section.2.OFFSET_IN_FILE .2.PointerToRelocations dd 0 .2.PointerToLineNumbers dd 0 .2.NumberOfRelocations dw 0 .2.NumberOfLineNumbers dw 0 .2.Characteristics dd 0x40000000 ; IMAGE_SCN_MEM_READ SectionTable.End:
The name of the section is stored in an 8-byte field, padded with zeros. We use DQ to define this as a 64-bit value and convert the string to a number with the + operator, in order to enable range check. A DQ with a string argument would allow text of any length and it would simply pad it so that the size was a multiple of 8 bytes. By converting text to a number we ensure that it has to fit in a single 64-bit cell so the field is always exactly 8 bytes long.
VirtualAddress and VirtualSize define the boundaries of a section within the image in memory. The starting address needs to be set up consistently with the SectionAlignment, we need to keep this in mind later when we define the labels used here.
PointerToRawData and SizeOfRawData define the placement of the contents of a section within the file. Both values have to be aligned accordingly to the FileAlignment, so it is possible for section's data in file to be larger than the size of that section in memory. It can also be the other way around, since a section may reserve more memory than it contains actual data. In an extreme case the size in file might be 0 when a section contains nothing but reserved memory. We are going to compute the constants used there with help of the $% symbol, after ensuring the proper alignment within the file.
The fields that refer to relocations and line numbers are in these structures because COFF objects use them, but for PE images they should be zeroed. Although PE could contain some relocations, they would be very different from the ones used by COFF and defined elsewhere (we are going to discuss them a bit later, the first example can work without them).
Characteristics contain various flags, here we just mark both sections as a readable memory and the code section as executable. These settings translate directly into the attributes of allocated memory pages, so they are quite important. We could also use values like IMAGE_SCN_CNT_CODE and IMAGE_SCN_CNT_INITIALIZED_DATA (connected to the fields like SizeOfCode and SizeOfInitializedData in the main header), but this would mostly be just decorative.
The end of the section table is also the end of the contents of the headers. Before we go further, we are going to fill up a few of the related constants. They are a bit redundant, the effect would be the same if we plugged the corresponding expressions directly in the places where we used their names earlier. But the use of middlemen constants helps to comfortably alter the way they are computed when this comes up in the future.
To count the number of records in a table we divide the total size by the length of a single entry as defined in the specification. As long as we define the tables correctly, everything should add up.
NUMBER_OF_RVA_AND_SIZES := (SectionTable-RvaAndSizes)/8 NUMBER_OF_SECTIONS := (SectionTable.End-SectionTable)/40 SIZE_OF_HEADERS := Section.1.OFFSET_IN_FILE
As for the total size of headers, it has to be rounded up to the nearest multiple of FILE_ALIGNMENT, and this is at the same time the position where the contents of the initial section is going to begin. Therefore we can cheat a little and shift the responsibility to another constant, the one defining the offset in file for the first section.
However, to correctly position our initial section we need to do some actual work.
First, we move in the image to the nearest multiple of SECTION_ALIGNMENT, by adding the right amount of reserved data (this is the default behavior of our ALIGN macro). This allows us to define the label corresponding to the start of the first section in memory.
align SECTION_ALIGNMENT Section.1:
Then we use the SECTION instruction of the assembler to cut off all the reserved bytes so they do not get included in file. In this particular case the only reserved bytes to discard are the ones made by the previous alignment.
section $%% align FILE_ALIGNMENT,0 Section.1.OFFSET_IN_FILE:
With the use of $%% as an argument to SECTION we temporarily switch from in-memory addressing to one tracing the actual position in file. This makes the address $ equal to the offset $% until we change this with another SECTION or ORG.
After that we use the alignment macro once more, this time to align the offset in file to the nearest multiple of FILE_ALIGNMENT. While the previous alignment just moved our address in memory without adding anything to file, this time we provide the second argument to the macro to make it write the necessary amount of zeroed bytes to the output.
Then Section.1.OFFSET_IN_FILE can be defined simply as a label, thanks to the address being the same as the position in file.
Finally we switch back to in-memory addressing, at the address of Section.1 label. A simple ORG would suffice, but we use SECTION for the visual appeal:
This is the entirety of our executable code, with entry point defined at the start of the section. There are just two types of x86 instructions used in this example, PUSH to store the arguments for the API functions on the stack, and CALL to execute the functions. In the next section we are going to set up the pointers to the functions and the character strings for MessageBoxA.
section Section.1 EntryPoint: push 0 push CaptionString push MessageString push 0 call [MessageBoxA] push 0 call [ExitProcess] Section.1.End:
Now we need to perform the full alignment ritual again, this time to set up the position of the second section. We also calculate the size of the first one in file simply by computing the difference between the aligned offsets.
With the proper alignments done, we move on to the second section, the one where we are going to put all the data.
align SECTION_ALIGNMENT Section.2: section $%% align FILE_ALIGNMENT,0 Section.1.SIZE_IN_FILE := $ - Section.1.OFFSET_IN_FILE Section.2.OFFSET_IN_FILE:
We start with the import table, which allows us to direct the loader to fill up our pointers with the addresses of the functions from system DLL files. This is actually a complex structure that consist of several smaller tables. First, there is an Import Directory Table.
Every record in this main table declares a single DLL file from which we want to import functions. The table ends with a record that has all five fields zeroed.
ImportTable: .1.ImportLookupTableRva dd KernelLookupTable-IMAGE_BASE .1.TimeDateStamp dd 0 .1.ForwarderChain dd 0 .1.NameRva dd KernelDLLName-IMAGE_BASE .1.ImportAddressTableRva dd KernelAddressTable-IMAGE_BASE .2.ImportLookupTableRva dd UserLookupTable-IMAGE_BASE .2.TimeDateStamp dd 0 .2.ForwarderChain dd 0 .2.NameRva dd UserDLLName-IMAGE_BASE .2.ImportAddressTableRva dd UserAddressTable-IMAGE_BASE dd 0,0,0,0,0
NameRva is a relative address of the name of DLL file. We are going to put these names near the end of the import-related data.
ImportLookupTableRva and ImportAddressTableRva point to two parallel tables. The former contains relative addresses of structures declaring functions to be imported, while the latter is going to contain actual addresses of imported functions. The functions can be in any order, as long as the same one is used for both tables. When our image is loaded into memory, the operating system is going to look for all the functions defined by the first table and fill the second one with corresponding addresses.
TimeDateStamp and ForwarderChain fields are used when the imports are bound - that is, when the second table is pre-filled with addresses of imported functions to save time when loading the image. This obviously can work correctly only when all the addresses in imported library are exactly as they were upon binding, and TimeDateStamp keeps the value of the timestamp of the DLL to provide a way to verify that it is exactly the same file. If the timestamps match, the loader can skip looking up all the functions, otherwise it does it as usual. Our imports are not bound, we need the loader to fill the addresses for us, therefore we keep TimeDateStamp zeroed in every case.
If the imports were bound, ForwarderChain would be interpreted as an index of a function that could not be bound because it was a forwarded import from another DLL. The value of the corresponding entry in the import address table would be an index of another such function, and so on. If we wanted to indicate that there were no such functions, we should put -1 in this field, but since we do not use binding (as indicated by the zeroed TimeDateStamp) this value is irrelevant.
Now we need to create lookup tables and address tables for every DLL. The initial contents of the parallel tables should be the same, they both should contain relative addresses to the lookup entries defining the functions. When the image is loaded, the IAT is rewritten with the matching addresses. We can then use these values directly, therefore we label them with names of the functions and this is exactly what is needed to get the CALL instructions in our code to work.
KernelLookupTable: dd ExitProcessLookup-IMAGE_BASE dd 0 KernelAddressTable: ExitProcess dd ExitProcessLookup-IMAGE_BASE ; this is going to be replaced with the address of the function dd 0 UserLookupTable: dd MessageBoxALookup-IMAGE_BASE dd 0 UserAddressTable: MessageBoxA dd MessageBoxALookup-IMAGE_BASE ; this is going to be replaced with the address of the function dd 0
We import only one function from each DLL, so the tables are short. The end of a table is marked by a zeroed entry.
Next come the lookup definitions for individual functions. Each such structure contains a 16-bit hint followed by the name of the function as a null-terminated string. The hint is an index into the export table of DLL, where the loader may look for the function with such name. If the hint fails, the loader continues to search for the function as usual, thus we do not have to know the right values to put there.
Even though this most often does not matter, the 16-bit values should be aligned to their "natural boundary" (that is their address should be a multiple of 2), while the string could end on an uneven address. For this reason we put an ALIGN between the records.
ExitProcessLookup: .Hint dw 0 .Name db 'ExitProcess',0 align 2 MessageBoxALookup: .Hint dw 0 .Name db 'MessageBoxA',0
Finally, we conclude the import table with the names of DLL files that we import. They are a plain null-terminated strings.
Since we are at it, we can define a couple more strings here. The import table has ended, but we can keep placing more data into the '.rdata' section and we still need to define the caption and the content for the message box that this program wants to show.
KernelDLLName db 'KERNEL32.DLL',0 UserDLLName db 'USER32.DLL',0 ImportTable.End:
This marks the end of our second section and, in fact, of the entire image. All that is left in another sequence of memory and file alignments.
CaptionString db "PE tutorial",0 MessageString db "I am alive and well!",0 Section.2.End:
This time there is no next section, so we do not define labels and constants that would refer to it. Instead we define SIZE_OF_IMAGE, which needed to be a multiple of SECTION_ALIGNMENT too.
align SECTION_ALIGNMENT SIZE_OF_IMAGE := $ - IMAGE_BASE section $%% align FILE_ALIGNMENT,0 Section.2.SIZE_IN_FILE := $ - Section.2.OFFSET_IN_FILE
This is it, the source for our first PE image is ready (a copy is in the attached "basic.asm" file). We can now assemble it into a file with the "exe" extension and let it run.
We can also combine it with the "listing.inc" script to contemplate the binary data juxtaposed with the commands that generated it. You may notice that numerous lines from "80386.inc" show up in the listing. To get rid of them, we can hide the included file inside a simple macro:
While we are at it, we may also incorporate the "ntimage.inc" file that defines some of the constants associated with PE format:
macro use? file* include file end macro use '80386.inc' use32
This allows easier experimentation with some of the values that earlier we hard-coded:
For example, we can add IMAGE_DLLCHARACTERISTICS_NX_COMPAT to DllCharacteristics, allowing to enable DEP (Data Execution Prevention).
.Characteristics dw IMAGE_FILE_32BIT_MACHINE + IMAGE_FILE_EXECUTABLE_IMAGE
This can make the IMAGE_SCN_MEM_EXECUTE bit in our section definitions really mean something.
.DllCharacteristics dw IMAGE_DLLCHARACTERISTICS_NX_COMPAT
It was a first step towards making our source more maintainable. Another one could be to automate some of the tasks. For example, we can generate all the entries in the section table with a simple repetition:
REPEAT allows to assemble the same piece of source in multiple copies and in every copy it replaces the name of the counter with the corresponding number. We defined a counter named "n" that starts from 1 and this generates the same labels as we had previously there.
SectionTable: repeat NUMBER_OF_SECTIONS, n:1 .n.Name dq Section.n.NAME .n.VirtualSize dd Section.n.End - Section.n .n.VirtualAddress dd Section.n - IMAGE_BASE .n.SizeOfRawData dd Section.n.SIZE_IN_FILE .n.PointerToRawData dd Section.n.OFFSET_IN_FILE .n.PointerToRelocations dd 0 .n.PointerToLineNumbers dd 0 .n.NumberOfRelocations dw 0 .n.NumberOfLineNumbers dw 0 .n.Characteristics dd Section.n.CHARACTERISTICS end repeat SectionTable.End:
This approach requires that we define several more constants. We also have to change how the NUMBER_OF_SECTIONS is defined, we can no longer compute it from the size of the section table, as this would create a circular dependence:
We can, however, take it a step further and automate everything by making a macro to define sections:
NUMBER_OF_SECTIONS := 2 Section.1.NAME := +'.text' Section.1.CHARACTERISTICS := IMAGE_SCN_MEM_EXECUTE + IMAGE_SCN_MEM_READ Section.2.NAME := +'.rdata' Section.2.CHARACTERISTICS := IMAGE_SCN_MEM_READ
We named the macro the same as the instruction of fasmg - this is allowed, and inside our macro the name still refers to the original instruction.
CURRENT_SECTION = 0 macro section? name*, characteristics:0 CURRENT_SECTION = CURRENT_SECTION + 1 repeat 1, new:CURRENT_SECTION, previous:CURRENT_SECTION-1 Section.previous.End: align SECTION_ALIGNMENT Section.new.NAME := +name Section.new.CHARACTERISTICS := characteristics Section.new: section $%% align FILE_ALIGNMENT,0 if previous > 0 Section.previous.SIZE_IN_FILE := $ - Section.previous.OFFSET_IN_FILE end if Section.new.OFFSET_IN_FILE: org Section.new end repeat end macro
To define labels and constants that correspond to enumerated section entries, we need to extract the number from the CURRENT_SECTION variable and somehow place it into names. The trick in fasmg is to use REPEAT with just a single repetition, solely for the purpose of defining counters that get replaced with numbers before the repeated text is assembled.
The macro does everything that we have previously done manually when starting a new section. The ending address and the size in file get defined only when the next section is started, so we need to define an additional false (not counted into the total number) section at the end, together with the definition of the NUMBER_OF_SECTIONS and the SIZE_OF_IMAGE.
With help of POSTPONE we can place this next to the definition of macro, for a better organization of source. Whatever is inside such block, gets assembled at the end of text.
postpone NUMBER_OF_SECTIONS := CURRENT_SECTION section '' SIZE_OF_IMAGE := $ - IMAGE_BASE end postpone
This macro required us to learn a bit more of the assembler's trickery, but it makes the section definitions much more pleasant to the eye:
Before we move on to learn about some other tables defined in the optional header, we may automate this set of definitions as well. Everything between RvaAndSizes and SectionTable labels can be replaced with the following construction:
section '.text', IMAGE_SCN_MEM_EXECUTE + IMAGE_SCN_MEM_READ EntryPoint: push 0 push CaptionString push MessageString push 0 call [MessageBoxA] push 0 call [ExitProcess] section '.rdata', IMAGE_SCN_MEM_READ ImportTable: .1.ImportLookupTableRva dd KernelLookupTable-IMAGE_BASE .1.TimeDateStamp dd 0 .1.ForwarderChain dd 0 .1.NameRva dd KernelDLLName-IMAGE_BASE .1.ImportAddressTableRva dd KernelAddressTable-IMAGE_BASE .2.ImportLookupTableRva dd UserLookupTable-IMAGE_BASE .2.TimeDateStamp dd 0 .2.ForwarderChain dd 0 .2.NameRva dd UserDLLName-IMAGE_BASE .2.ImportAddressTableRva dd UserAddressTable-IMAGE_BASE dd 0,0,0,0,0 KernelLookupTable: dd ExitProcessLookup-IMAGE_BASE dd 0 KernelAddressTable: ExitProcess dd ExitProcessLookup-IMAGE_BASE ; this is going to be replaced with the address of the function dd 0 UserLookupTable: dd MessageBoxALookup-IMAGE_BASE dd 0 UserAddressTable: MessageBoxA dd MessageBoxALookup-IMAGE_BASE ; this is going to be replaced with the address of the function dd 0 align 2 ExitProcessLookup: .Hint dw 0 .Name db 'ExitProcess',0 align 2 MessageBoxALookup: .Hint dw 0 .Name db 'MessageBoxA',0 KernelDLLName db 'KERNEL32.DLL',0 UserDLLName db 'USER32.DLL',0 ImportTable.End: CaptionString db "PE tutorial",0 MessageString db "I am alive and well!",0
ITERATE assembles a block as many times as there are items on the list, substituting the text of a consecutive item for a name given in the first argument (the items start from the second one). Inside it is tested whether a symbol with a name like ExportTable or ImportTable is defined anywhere and only in such case the fields are filled. At the moment we only have ImportTable present, but as a soon as we add another one of the listed, the optional header is going to contain the right values to make it work.
iterate name, Export, Import, Resource, Exception, Certificate, BaseRelocation, Debug, Architecture, GlobalPtr, TLS, LoadConfig, BoundImport, IAT, DelayImport, COMPlus, Reserved if defined name#Table .name.Rva dd name#Table-IMAGE_BASE .name.Size dd name#Table.End-name#Table else .name.Rva dd 0 .name.Size dd 0 end if end iterate
A variant of the first source that has all these improvements is in the attached "basic_template.asm" file. We are going to use it as a base for the continued experiments.
Last edited by Tomasz Grysztar on 16 Aug 2018, 08:52; edited 6 times in total
|04 Aug 2018, 23:44||
1.2 Adding relocations
As we have seen in the first sample, a PE image has a chosen base address at which it is constructed. The headers and additional tables of PE format are not really affected by this choice - whenever they point to something they do it through a relative address (RVA), which is independent of the selection of base. But the instructions of the program use the actual addresses and therefore they require the image to be loaded exactly where expected.
Nevertheless, the operating system may need or want to load the image at a different base. The address may be unavailable for some reason (though this should not happen when the recommended base has been used). When the PE file is a DLL loaded into memory of a process, its chosen address may already be occupied by another image (either the main program or a different library). Furthermore, with ASLR (Address space layout randomization), the system may prefer to load image at a random address as a security precaution.
In all these cases the loader needs a way to adjust the instructions of the program so that they refer to the correct addresses. The image may provide information how to do it, in form of a relocation table.
Before we learn how to construct such table, we should take a look at what needs to be adjusted when the base address can change. We are going to use the instruction of fasmg called ELEMENT, which defines a special kind of symbol that can be used like an address, but does not have a fixed value.
Let us experiment with the previously prepared template and expand the couple of lines that set up the base address.
With this modification, IMAGE_BASE now contains a variable term (called BASE_RELOCATION) and thanks to ORG all the labels referring to memory are going to have it, too.
DEFAULT_IMAGE_BASE := 0x400000 element BASE_RELOCATION IMAGE_BASE := DEFAULT_IMAGE_BASE + BASE_RELOCATION org IMAGE_BASE
We should now try to assemble it. The assembly is not going to succeed, but the errors we get may give us hints at what to do next.
The first error points to ALIGN macro. Obviously, when the value of $ is an address than contains a variable term, the assembler is not able to evaluate the expression that computes the distance to the next aligned address. We can get around it with a simple modification to the macro.
With help of RELATIVETO operator we detect whether the $ is an address with this variable and then we compute the same expression but with BASE_RELOCATION subtracted from $. According to the specification, the address of an image should always be a multiple of 65536. Therefore the addresses aligned to any power of two up to that number should stay aligned when the base is moved.
macro align? pow2*,value:? if $ relativeto BASE_RELOCATION db (BASE_RELOCATION-$)and(pow2-1) dup value else db (-$)and(pow2-1) dup value end if end macro
To keep the macro simple, we have not been making it verify whether the received number is really a power of two. Therefore we are also not making it check if the number is not larger than 65536 when the address has a variable base, but we should keep this restriction in mind. After gaining more experience with fasmg it should be easy to come back to this macro and make it more resistant to misuse.
We got rid of the errors caused by ALIGN, the next one we should see is the declaration of ImageBase. It requires a really simple correction, this field should actually have the value of our default base:
.ImageBase dd DEFAULT_IMAGE_BASE
The other fields in headers do not cause any errors. When an RVA is computed, it is a difference between an address in memory and IMAGE_BASE. Both these values contain the same variable term (BASE_RELOCATION), so subtracting one from the other results in just a plain number and everything works out.
At this moment the only remaining errors in the assembly are going to be caused by the program instructions. And this is where we get to the core of the problem.
The error message may look like this:
PUSH is implemented as a macro, and we can see that internally it uses DD to put a 32-bit value somewhere inside the generated instruction code. It fails because of the variable term we introduced, so that value is certainly an address.
flat assembler version g.i97rp test.asm : push CaptionString macro push  macro push_instruction : dd @src.imm Processed: dd @src.imm Error: variable term used where not expected.
We already know that we can override the instructions of the assembler with macros. It should be possible to replace DD so that it could deal with the problem somehow.
When DD receives a movable value, we can subtract BASE_RELOCATION and obtain a plain number. This result would be a correct address as long as the image was loaded at its default base. We are going to store this address in the code of an instruction as usual, but we also need to let the loader know that if the base address is different, this value should be adjusted accordingly.
To generate this kind of information, we are going to collect the relative addresses of all values generated with DD that need to be corrected when the base is moved.
The outer layers of this macro are dedicated to handling the syntax of DD. The only argument to the macro is modified with an ampersand to signalize that it should catch everything that follows DD instruction. This may contain multiple values separated with commas, so we then iterate through all of them. We need to be able to handle a question mark in place of a value, so we detect it with MATCH (which is like a special form of IF, at least in this situation).
macro dd? data& iterate unit, data match ?, unit dd ? else if unit relativeto BASE_RELOCATION repeat 1, i:FIXUP_INDEX FIXUP_RVA_#i := $ - IMAGE_BASE end repeat FIXUP_INDEX = FIXUP_INDEX + 1 dd unit-BASE_RELOCATION else dd unit end if end iterate end macro
When the unit of data is a movable address, we subtract BASE_RELOCATION from the value before passing it to the original DD. But before doing that, we collect the relative address of this place in code and store it in a numbered constant.
The variable that we use to count the constants needs to be initialized first. And at the end of source we are going to put the final number of collected addresses into yet another constant:
We have the information gathered, now we need to put it in a form that the loader can understand.
FIXUP_INDEX = 0 postpone FIXUP_COUNT := FIXUP_INDEX end postpone
First, we should add another section specifically for this data. The macros that we prepared earlier make it easy.
We used a new attribute, one that describes the section as discardable. After the loader uses the information to adjust the code to the new address of residence, the content of this section is no longer going to be needed.
section '.reloc', IMAGE_SCN_MEM_READ + IMAGE_SCN_MEM_DISCARDABLE
Now the most important part, the actual table defining all the places in code that require adjustments. We need to begin it with BaseRelocationTable label so that it becomes included in the right place in the optional header.
The base relocation table is divided into blocks, each one contaning entries corresponding to addresses within a single 4096-byte page. This allows to pack the entries tightly. The upper part of the address is common to all the entries in the block and is stored in one place before them. The individual addresses stored within a block differ only in the low part and it suffices to store the bottom 12 bits of each one.
BaseRelocationTable: FIRST = 0 BASE = 0 repeat FIXUP_COUNT+1, INDEX:0 if INDEX = FIXUP_COUNT | FIXUP_RVA_#INDEX and not 0xFFF <> BASE and not 0xFFF if INDEX = FIXUP_COUNT | INDEX > FIRST ALIGNMENT = (INDEX-FIRST) and 1 dd BASE dd 8+(INDEX-FIRST+ALIGNMENT)*2 repeat (INDEX-FIRST), i:FIRST dw FIXUP_RVA_#i and 0xFFF + IMAGE_REL_BASED_HIGHLOW shl 12 end repeat if ALIGNMENT dw 0 end if end if if INDEX < FIXUP_COUNT FIRST = INDEX BASE = FIXUP_RVA_#INDEX and not 0xFFF end if end if end repeat BaseRelocationTable.End:
To generate such structure, we iterate through all the collected values and wait until the upper part of the address differs from the previous one stored in BASE variable. When this happens, we make a block with all the addresses that had common upper part up to this point, set up FIRST to be the number of the first address for the next block and update BASE with the new value.
We perform one additional iteration, to make the last block. This is why the conditions contain additional clauses to check if it is a final round and the block is then forcibly generated.
Each block starts with two 32-bit values, first is the base address for all entries (with the low 12 bits cleared), second is the total length of the block. Every entry that follows is 2 bytes long, but the length of a block needs to be a multiple of 4, so if the number of entries is odd, we need to add a dummy entry at the end. To do this, we define ALIGNMENT by taking the lowest bit of the number of entries, so it is 1 when there is an odd number of entries and 0 otherwise.
An entry inside a block is a 16-bit value, with the low 12 bits being the bottom part of the address, and the upper bits containing a description of a method that should be used to adjust the code at that address. The dummy entries have these bits zeroed and it corresponds to a "do nothing" method. In the regular entries we universally choose IMAGE_REL_BASED_HIGHLOW. This makes the loader adjust a 32-bit value in code by adding to it the difference between the new base address and the default one. It is exactly what we need to correct the values generated by our DD macro.
If this seems a bit complex, it might be a good idea to use our "listing.inc" and look at what is generated by this part of the source:
 BaseRelocationTable:  FIRST = 0  BASE = 0  FIRST = 0  BASE = FIXUP_RVA_#0 and not 0xFFF  ALIGNMENT = (4-FIRST) and 1  00000600: 00 10 00 00 dd BASE  00000604: 10 00 00 00 dd 8+(4-FIRST+ALIGNMENT)*2  00000608: 03 30 dw FIXUP_RVA_#0 and 0xFFF + IMAGE_REL_BASED_HIGHLOW shl 12 [000000000040300A] 0000060A: 08 30 dw FIXUP_RVA_#1 and 0xFFF + IMAGE_REL_BASED_HIGHLOW shl 12 [000000000040300C] 0000060C: 10 30 dw FIXUP_RVA_#2 and 0xFFF + IMAGE_REL_BASED_HIGHLOW shl 12 [000000000040300E] 0000060E: 18 30 dw FIXUP_RVA_#3 and 0xFFF + IMAGE_REL_BASED_HIGHLOW shl 12  BaseRelocationTable.End:
Now the base relocation table is there for the loader to use, but we should also explicitly state that our executable does not mind being loaded at an address different from default. We do this by adding IMAGE_DLLCHARACTERISTICS_DYNAMIC_BASE to DllCharacteristics.
With this bit set, the image can participate in ASLR when the system supports it.
.DllCharacteristics dw IMAGE_DLLCHARACTERISTICS_NX_COMPAT + IMAGE_DLLCHARACTERISTICS_DYNAMIC_BASE
But how do we know that this worked? When we assemble such modified example and run it, it is going to display the same message as always, we do not know if it was loaded at the default base or some other address.
There is a simple method to ensure that the relocations are used. Windows cannot load an image at the address zero, this area is reserved by the system. If we generate PE file with such default base, the loader will not have a choice but to relocate. Therefore if we change DEFAULT_IMAGE_BASE to zero and the program keeps running as usual, we can treat it as a proof that the relocations function correctly.
Last edited by Tomasz Grysztar on 08 Aug 2018, 09:03; edited 1 time in total
|08 Aug 2018, 09:00||
1.3 Making a library
We already know how to import functions from a DLL, now it is time to make our own one. The relocatable image that we have just prepared should be our starting point. It is easy for libraries to have clashing addresses, even if we try to come up with a unique one, therefore having a relocation table is practically mandatory.
What distinguishes a DLL is a single bit in Characteristics, with a self-explanatory name:
But for our library to be of some use, we should make it contain some function that a program can then import and call.
.Characteristics dw IMAGE_FILE_32BIT_MACHINE + IMAGE_FILE_EXECUTABLE_IMAGE + IMAGE_FILE_DLL
Let us rewrite the '.text' section then.
The entry point of a DLL is a function itself, always called when a library has just been loaded, but also in few other situations. This function should return TRUE to indicate a success and this is done by loading 1 into the register EAX. We end the function with RET, but we also have to use its optional argument to clean 12 bytes off the stack. In 32-bit Windows the standard calling convention requires that the function cleans up the stack upon returning. The entry point function of a DLL takes three parameters, each of them is 32-bit and thus occupies 4 bytes, this gives the total of 12 bytes.
section '.text', IMAGE_SCN_MEM_EXECUTE + IMAGE_SCN_MEM_READ EntryPoint: mov eax,1 ret 12
What comes next is the function that we are going to export.
It does not take any parameters, does the same thing that all our previous examples did and then returns.
ShowOff: push 0 push CaptionString push MessageString push 0 call [MessageBoxA] ret
The function is ready, but now we need to construct an export table for our DLL. We can put in it the '.rdata' section, just like the import table, we only need to make sure that the starting address is aligned. A table that lies right at the start of the section inherits a nice alignment, but when we put one on top of another data, we should better use ALIGN to keep the address round enough.
The initial part is the Export Directory Table, which always contains just one record, as it exports function of a just a single library.
align 4 ExportTable: .ExportFlags dd 0 .TimeDateStamp dd %t .MajorVersion dw 0 .MinorVersion dw 0 .NameRva dd LibraryName-IMAGE_BASE .OrdinalBase dd 1 .AddressTableEntries dd 1 .NumberOfNamePointers dd 1 .ExportAddressTableRva dd ExportAddressTable-IMAGE_BASE .NamePointerRva dd ExportNamePointerTable-IMAGE_BASE .OrdinalTableRva dd ExportOrdinalTable-IMAGE_BASE
ExportFlags is an unused field and should always be zero. TimeDateStamp is the value used to verify whether the bound imports refer to the same version the library, as mentioned earlier. Nowadays ASLR makes bound imports mostly obsolete and we are not going to use them, but we put a good timestamp there just in case someone wanted to try binding to our library.
MajorVersion and MinorVersion we can set to any values we wish, they are mostly irrelevant.
NameRva points the name of the library, a string that we are going to define near the end of the table to not mess up the alignment.
To understand the purpose of OrdinalBase we need to discuss one thing about the import table that got omitted earlier. It possible to specify a function to import not by the name, but by an ordinal number which identifies a record within the Export Address Table. The addresses (and therefore the functions they point to) are numbered starting from the value of OrdinalBase. A usual choice of this offset is 1, then the first address in the table has ordinal 1, the second one - ordinal 2, and so on.
An entry in Import Lookup Table that would normally contain an RVA may be marked as containing an ordinal number by having the highest bit set. Its value is then 0x8000000 plus the ordinal of a function to import. However, this is a discouraged method. Different versions of a library may not have the same numbering of the functions unless someone paid a close attention to it. Therefore we are not going to use this technique and the value of OrdinalBase has no other uses.
AddressTableEntries gives the number of addresses in EAT, which has a relative address specified in ExportAddressTableRva.
NumberOfNamePointers defines the number of entries in two other tables, pointed to by NamePointerRva and OrdinalTableRva. These tables run in parallel, the records of the first one point to the names of the functions while the second one defines corresponding indexes in EAT. The latter are 16-bit numbers that count the records in EAT starting from zero and do not depend on the contents of OrdinalBase.
EAT can have a different length than the other two tables. It is possible to have addresses that have no associated name and such functions could be imported only through their ordinal numbers. On the other hand it is also possible to have multiple names pointing to the same entry in EAT so a single function can have several aliases.
In this example we define just one function with one name, thus we put 1 everywhere.
The entry in EAT is the relative address of the exported function. This seems pretty straightforward, but there is a small subtlety hidden here. The address is treated as pointing directly to a function as long as it does not lie within the boundaries of the export table (in our case between ExportTable and ExportTable.End labels), otherwise it would be interpreted as the address of a string describing the function in another library. This is how the forwarded exports can be made. However in this example we just export our own function.
ExportAddressTable: dd ShowOff-IMAGE_BASE
The parallel tables connect the addresses of strings with the indexes into EAT. The addresses are counted from zero so this is the number we need to associate with the exported name of our only function. The names in the table need to be ordered lexically (to allow an efficient binary search) but when we have just one this is not an issue.
ExportNamePointerTable: dd ShowOff.Name-IMAGE_BASE ExportOrdinalTable: dw 0
All that remains is to define the needed strings, at this point we no longer have to worry about them causing any misalignment.
The export table is therefore complete and our library should be ready to use. As a final touch, we may alter the message that would be shown by the function, so when we see it during testing we should know that it really comes from this freshly made DLL:
LibraryName db 'LIBRARY.DLL',0 ShowOff.Name db 'ShowOff',0 ExportTable.End:
MessageString db "This is a message from the depth of the library.",0
But to test the library, we also need to make a program that uses it. We should once more copy our template and modify the contents of the '.text' section to make it call the function from our DLL:
Of course we also need to modify the import table to have this function actually available. This should be easy now that we know what goes where, but doing it manually can quickly become a dull task (even if it might be a good exercise at first). We should make a little investment and prepare a macro that would do most of the boring things for us.
EntryPoint: call [ShowOff] push 0 call [ExitProcess]
This macro interates multiple times through a list of items and it uses MATCH to distinguish two kinds of them. If one ends with ".DLL" it is taken as a name of a library to import, other items are the names of functions belonging to a DLL specified just before them.
macro import? items& align 4 ImportTable: iterate item, items match name.=DLL?, item .name.ImportLookupTableRva dd ImportLookupTable.name-IMAGE_BASE .name.TimeDateStamp dd 0 .name.ForwarderChain dd 0 .name.NameRva dd ImportLibraryName.name-IMAGE_BASE .name.ImportAddressTableRva dd ImportAddressTable.name-IMAGE_BASE else if % = 1 err 'please start with a name of a DLL' end if end iterate dd 0,0,0,0 iterate item, items match name.=DLL?, item dd 0 ImportLookupTable.name: else dd ImportLookup.item-IMAGE_BASE end match end iterate iterate item, items match name.=DLL?, item dd 0 ImportAddressTable.name: else item dd ImportLookup.item-IMAGE_BASE end match end iterate dd 0 iterate item, items match name.=DLL?, item ImportLibraryName.name db `item,0 else align 2 ImportLookup.item: dw 0 db `item,0 end match end iterate ImportTable.End: end macro
A few new tricks have been used here. When a pattern given to MATCH contains a name that is not preceded by equality sign, it is a wildcard matching any non-empty text. Moreover, that name is then replaced with the corresponding text everywhere inside the MATCH block. In this case "name" becomes a parameter containing the name of the library without the extension. The "dll" text is matched literally because of the equality sign, though the trailing question mark makes this a case-insensitive requirement.
The macro should not allow function names to be given without a library being defined first, so if the initial item does match the pattern, an error is signalled. The number of processed item is taken from the special counter %, which is available in any repeating block.
The first loop looks only at the names of the libraries and defines Import Directory Table with records for each one of them. The table should end with five zeroed fields, but it only makes four of them. This is a little tricky - in the following loops every time a new ILT or IAT is started, it generates a zeroed field to close the previous one. The very first time it happens, that zero becomes the missing fifth one to complete the Import Directory Table.
When it comes to defining the names of libraries and function, the backquote is used to make the text of a parameter into a string.
The internal workings of the macro are normally hidden when we generate listing, but there is a way to prioritize the expansion of the macro and make the lines it generates show up. An exclamation mark after the name gives macro such priority:
This is a trick to keep in mind for whenever a more detailed listing might be helpful.
macro import?! items&
With the assistance of the macro we can now define an entire '.rdata' section for our testing program in a just a couple of lines:
The backslash character is used to break a single line into multiple ones, solely for readability.
section '.rdata', IMAGE_SCN_MEM_READ import LIBRARY.DLL, ShowOff, \ KERNEL32.DLL, ExitProcess
We can now assemble the program and the library, keeping in mind that the latter should be written into a file named "library.dll". The attached files "library.asm" and "library_user.asm" contain sources prepared according to the above process.
If we keep experimenting, we might want to create libraries with more functions. This is a good excuse to create another macro, we should not have to concern ourselves with manual creation of the export tables.
A syntax for this macro should be analogous to the one we have for imports. In this case there is just one library to talk about, so the name of the DLL comes as the first argument and all that follows are the names of the functions.
macro export? library*,functions& align 4 iterate name, functions EXPORT_ADDRESS.% := name-IMAGE_BASE EXPORT_NAME.% = `name EXPORT_ORDINAL.% = %-1 EXPORT_COUNT = %% end iterate
We define a numbered constants with values that should go into appropriate sub-tables. We need them in this form, because we may have to shuffle some around, remembering that the table of names must have them ordered lexically.
We also define EXPORT_COUNT value using a special symbol %%, which is a companion to % and is the total number of repetitions. The same definition is redone with every iteration, but this is quite harmless. Actually, that %% could be replaced with % to the same effect, as only the last assigned value counts.
This portion of the macro performs a classic binary Shell sort algorithm to order the list of names. If the functions given to the macro are already in lexical order, this block does nothing.
D = EXPORT_COUNT while D > 1 D = D shr 1 repeat EXPORT_COUNT-D X = D+% while X-D > 0 repeat 1, i:X-D, j:X if lengthof EXPORT_NAME.i > lengthof EXPORT_NAME.j S = lengthof EXPORT_NAME.i else S = lengthof EXPORT_NAME.j end if if EXPORT_NAME.i bswap S > EXPORT_NAME.j bswap S T = EXPORT_NAME.i EXPORT_NAME.i = EXPORT_NAME.j EXPORT_NAME.j = T T = EXPORT_ORDINAL.i EXPORT_ORDINAL.i = EXPORT_ORDINAL.j EXPORT_ORDINAL.j = T else break end if end repeat X = X-D end while end repeat end while
While the outer layers should more or less speak for themselves, the interior is a bit convoluted because of the peculiarities of the language. The innermost REPEAT is not a real loop but an idiomatic expression that we have already seen before. It makes a text substitution that replaces "i" and "j" with numbers computed from associated expressions.
The assembler performs comparisons numerically and the strings are converted into numbers using the little-endian encoding - the first byte of the text is the least significant byte of the number. Therefore to compare texts lexically we need to reverse the order of bytes in corresponding numbers and this is done with BSWAP. The size of the reversed numeric data should be the same for both strings, because then compared values have the same position of the most significant byte (which contains what was the first byte of the text). The greater of the two lengths becomes the chosen size, since the numbers must be large enough to contain the strings.
When the list of names has some of its values swapped during the sorting, the corresponding ordinal numbers are exchanged as well, while the list of addresses remains unaltered. This way every function keeps the ordinal number it was assigned based on the order of names given to the macro.
Finally we build the actual tables. All the lists were constructed and sorted earlier, here they are just put into familiar structures.
ExportTable: .ExportFlags dd 0 .TimeDateStamp dd %t .MajorVersion dw 0 .MinorVersion dw 0 .NameRva dd ExportLibraryName-IMAGE_BASE .OrdinalBase dd 1 .AddressTableEntries dd EXPORT_COUNT .NumberOfNamePointers dd EXPORT_COUNT .ExportAddressTableRva dd ExportAddressTable-IMAGE_BASE .NamePointerRva dd ExportNamePointerTable-IMAGE_BASE .OrdinalTableRva dd ExportOrdinalTable-IMAGE_BASE ExportAddressTable: repeat EXPORT_COUNT dd EXPORT_ADDRESS.% end repeat ExportNamePointerTable: repeat EXPORT_COUNT dd ExportName.%-IMAGE_BASE end repeat ExportOrdinalTable: repeat EXPORT_COUNT dw EXPORT_ORDINAL.% end repeat ExportLibraryName db `library,0 repeat EXPORT_COUNT ExportName.% db EXPORT_NAME.%,0 end repeat ExportTable.End: end macro
With such defined macro the export table of our library could be defined simply as:
A source text that uses all the macros we have made so far can be found in the attached "library_template.asm".
export LIBRARY.DLL, ShowOff
Last edited by Tomasz Grysztar on 16 Aug 2018, 08:58; edited 14 times in total
|08 Aug 2018, 09:03||
1.4 Embedding resources
To be continued...
ResourceTable: .Characteristics dd 0 .TimeDateStamp dd 0 .MajorVersion dw 0 .MinorVersion dw 0 .NumberOfNameEntries dw 0 .NumberOfIdEntries dw 1 .1.Id dd RT_MANIFEST .1.Offset dd 0x80000000 + ResourceDirectory_Manifest-ResourceTable ResourceDirectory_Manifest: .Characteristics dd 0 .TimeDateStamp dd 0 .MajorVersion dw 0 .MinorVersion dw 0 .NumberOfNameEntries dw 0 .NumberOfIdEntries dw 1 .1.Id dd CREATEPROCESS_MANIFEST_RESOURCE_ID .1.Offset dd 0x80000000 + ResourceDirectory_Manifest_CreateProcess-ResourceTable ResourceDirectory_Manifest_CreateProcess: .Characteristics dd 0 .TimeDateStamp dd 0 .MajorVersion dw 0 .MinorVersion dw 0 .NumberOfNameEntries dw 0 .NumberOfIdEntries dw 1 .1.Id dd LANG_NEUTRAL .1.Offset dd ResourceDataEntry_Manifest_CreateProcess-ResourceTable ResourceDataEntry_Manifest_CreateProcess: .DataRva dd Manifest_CreateProcess-IMAGE_BASE .Size dd Manifest_CreateProcess.End-Manifest_CreateProcess .Codepage dd 0 .Reserved dd 0 Manifest_CreateProcess: file 'manifest.xml' .End: ResourceTable.End:
Last edited by Tomasz Grysztar on 16 Aug 2018, 11:36; edited 4 times in total
|08 Aug 2018, 09:18||
Snippets for later use
postpone ? CHECKSUM = 0 repeat $% shr 1, POSITION:0 load H:word from :POSITION shl 1 CHECKSUM = CHECKSUM + H end repeat while CHECKSUM shr 16 CHECKSUM = CHECKSUM shr 16 + CHECKSUM and 0FFFFh end while CHECKSUM = CHECKSUM + $% store CHECKSUM:dword at :OptionalHeader.CheckSum-IMAGE_BASE end postpone
Alteration of section macro that correctly handles non-standard alignments
if SECTION_ALIGNMENT < 0x1000 assert FILE_ALIGNMENT = SECTION_ALIGNMENT org $%% else section $%% end if align FILE_ALIGNMENT,0
|08 Aug 2018, 12:46||
|16 Aug 2018, 09:08||
< Last Thread | Next Thread >
Copyright © 1999-2018, Tomasz Grysztar.
Powered by rwasa.