flat assembler
Message board for the users of flat assembler.

Index > Compiler Internals > PE64 DLL without fixups bug

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
>_<



Joined: 08 Apr 2015
Posts: 8
>_<
PE64 dll without "fixups", fasm set IMAGE_FILE_RELOCS_STRIPPED
after dll can't load if one of already loaded dlls have same base.. when i manually uncheck this flag - load normal with rand base...
so this is bug?
and i know about:
data fixups
end data
but that set additionally "Relocation Directory" address with ZERO size in "Data Directory" - looks dirty or another bug..
ps i test other linker, he not set IMAGE_FILE_RELOCS_STRIPPED when no reloc
Post 19 Aug 2015, 22:22
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
>_<
Quote:
PE64 dll without "fixups", fasm set IMAGE_FILE_RELOCS_STRIPPED

Yes, because you don't add data fixups. Not a bug.

Quote:
but that set additionally "Relocation Directory" address with ZERO size in "Data Directory"

Yes, because if you don't have anything to apply fixups to, then no fixups are generated, and hence zero-sized data directory. Not a bug.

_________________
Faith is a superposition of knowledge and fallacy
Post 19 Aug 2015, 23:04
View user's profile Send private message Reply with quote
>_<



Joined: 08 Apr 2015
Posts: 8
>_<
l_inc

1
there "rip relative" it can't use fixups. that flag block change imagebse. same effect if imagebase 0
look: https://msdn.microsoft.com/en-us/library/windows/desktop/ms680313%28v=vs.85%29.aspx

2
BaseRelocation address NOT ZERO. only size zero. example:
data fixups
end data
qqqqqq db 'test',0

Relocation address in "Data Directory" will point "test" string... u think that string looks as ZERO fixups table? xD
Post 20 Aug 2015, 07:00
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
>_<
Quote:
there "rip relative" it can't use fixups

Not everything in 64 bit code is rip-relative. E.g., mov to rax can use absolute addressing, in-memory stored addresses cannot be rip-relative, address values can be loaded into registers directly etc. All these cases require fixups. For that reason fixups still exist in 64-bit binaries.

Quote:
Relocation address in "Data Directory" will point "test" string...

That is correct. You specified it to be at that location. And as long as there's nothing to fixup the relocation table size is zero.

Quote:
u think that string looks as ZERO fixups table?

Sure. Its size is zero, so its address equals to the address of the following non-zero sized object. There's no problem here. And btw. there's no problem here for the windows loader either.

_________________
Faith is a superposition of knowledge and fallacy
Post 20 Aug 2015, 17:31
View user's profile Send private message Reply with quote
>_<



Joined: 08 Apr 2015
Posts: 8
>_<
yeye.. for windows maybe, but for other "special loaders" which check only address - crash.
always manually uncheck IMAGE_FILE_RELOCS_STRIPPED + set IMAGE_DLLCHARACTERISTICS_DYNAMIC_BASE or use (hm fixups without fixups? lol)
data fixups
end data
and after manually ZERO that address... u really think this is normal and user-friendly?
Post 20 Aug 2015, 19:21
View user's profile Send private message Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
>_<
Quote:
but for other "special loaders" which check only address - crash

That's a clear bug in the "special loader". In fact it makes much more sense to check the size than the rva, cause rva of a data directory could theoretically be zero with that data directory to be overlapped with the dos header. The size of zero clearly states that there's nothing in there to process.
Quote:
or use (hm fixups without fixups? lol)

If it's necessary for you to have fixups, then use something in your code what requires a fixup. E.g., dq $ .
Quote:
u really think this is normal and user-friendly?

No, I think both ideas are stupid. IMAGE_FILE_RELOCS_STRIPPED == 0 means there is a relocation directory. And that's exactly what you tell fasm by specifying data fixups. The relocation directory must have some address. It doesn't matter however what address this is as long as its size is zero.


A thing I find strange about fasm's behaviour here is that the data directive ignores address overriding. E.g. it should be possible for your case to specify org $-rva $ in front of the data directive, so that the rva of the data directory in the optional header is set to 0. This might be a bug indeed.

_________________
Faith is a superposition of knowledge and fallacy
Post 20 Aug 2015, 20:30
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7725
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
A thing I find strange about fasm's behaviour here is that the data directive ignores address overriding. E.g. it should be possible for your case to specify org $-rva $ in front of the data directive, so that the rva of the data directory in the optional header is set to 0. This might be a bug indeed.
fasm's ORG directive does not affect the output pointer, unlike TASM's one (as far as I know, fasm behaves like NASM in this case). With ORG you only change the assumed address for a given block of code or data, and it expects that this fragment will be loaded or later moved into memory area with such an address. So with ORG placed in front of DATA directive you can change the assumed address that is used to generate label values, but it does not change the pointer within output image where the data is put. The data directory contains an RVA which is an offset withing image, not affected by ORG setting.

As for the empty fixups, it is a part of a set of problems that were discussed many times in the past and these discussions are already mentioned by Windows-related FAQ. As for the handling of IMAGE_FILE_RELOCS_STRIPPED, you may find some additional explanation in my comment about the related bugfix.
Post 23 Aug 2015, 11:36
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
With ORG you only change the assumed address for a given block of code or data, and it expects that this fragment will be loaded or later moved into memory area with such an address.

That is absolutely clear.
Quote:
So with ORG placed in front of DATA directive you can change the assumed address that is used to generate label values, but it does not change the pointer within output image where the data is put.

Sure.
Quote:
The data directory contains an RVA which is an offset withing image

I can't agree with this one. RVA is an in-memory offset. "An offset within image" is a file pointer. There's only one data directory that requires a file pointer: certificate table. Which btw. justifies usefulness of a file pointer operator.
Quote:
not affected by ORG setting

The org setting should affect the assumed VA. And RVA is an offset directly derived from the assumed VA and assumed image base:
Code:
org $-rva $
;assumed VA is image base, hence assumed RVA is 0
data fixups
end data    

This is the same as with the entry point: you can put a label referenced by the entry directive say start after changing the assumed VA by the org directive. The entry point RVA that is then put into the optional header is the assumed VA of the start label less assumed image base.

_________________
Faith is a superposition of knowledge and fallacy
Post 23 Aug 2015, 12:22
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7725
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
I can't agree with this one. RVA is an in-memory offset. "An offset within image" is a file pointer. There's only one data directory that requires a file pointer: certificate table. Which btw. justifies usefulness of a file pointer operator.
RVA is an offset within the "memory image" as mentioned by PE/COFF specification. There is an exact correspondence between offsets within image and offsets within file. You cannot change one without changing the other.

l_inc wrote:
The org setting should affect the assumed VA. And RVA is an offset directly derived from the assumed VA and assumed image base (...)
As I see it, it is VA that is derived from RVA, not the other way around. And since VA is just a regular address within the memory, you are then free to move it around. But RVAs are used by the loader and they mean the offsets within the image as constructed from file. There is no such freedom here - you have no window of opportunity to move your data inside the constructed image during the loading process.

If you wanted to insert some number of your choice as an RVA of data directory, then some other directive (a different variant of DATA, perhaps similar to ENTRY) would have to be created for such purpose. Current "data"+"end data" implementation declares a part of file that corresponds uniquely to a part of image and it is not possible to change the RVA of that block of data without changing its offset in file or affecting the structure of file in some other way.
Post 23 Aug 2015, 13:02
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
RVA is an offset within the "memory image" as mentioned by PE/COFF specification

I'm not sure, what specification you have. I have 8.3 and I don't find this wording there. The definition of RVA according to the revision I have:
Microsoft PE and COFF Specification. General Concepts wrote:
the address of an item after it is loaded into memory, with the base address of the image file subtracted from it

Quote:
There is an exact correspondence between offsets within image and offsets within file. You cannot change one without changing the other.

I think you're continuously using the word "image" incorrectly. In the specification it's always referred to as a file, from which the program is loaded, not as an image of the on-disk "preimage" in a mathematical sense. And btw. same image location can be loaded at different RVAs during the standard image loading process.
Quote:
As I see it, it is VA that is derived from RVA, not the other way around

Well, this contradicts to the definition of RVA I cited above.
Quote:
There is no such freedom here - you have no window of opportunity to move your data inside the constructed image during the loading process.

Are we talking about definitions or about a specific implementation? There are DLL's, there are TLS callbacks, there are APC's and other means to affect the loading process.

_________________
Faith is a superposition of knowledge and fallacy
Post 23 Aug 2015, 13:59
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7725
Location: Kraków, Poland
Tomasz Grysztar
The newer specifications started to use the term "image file" when talking about the PE files, but I think that the "image" term by itself referred to the mapping of program or library into memory, and it was often used in this meaning in various MS documents. The terms like "image base" and "image size" that are still sometimes used in current specification are the remnants of this terminology.

The very old PE specification that was the only one I used when implementing PE output into fasm back in 1999/2000 contained definitions like:
Michael J. O'Leary in 'Portable Executable Format' wrote:
A VA is a virtual address that is already biased by the Image Base found in the PE Header. A RVA is a virtual address that is relative to the Image Base.
(...)
IMAGE SIZE = DD The virtual size (in bytes) of the image. This includes all headers. The total image size must be a multiple of Object Align.
Since this was the document that fasm's implementation was initially based on, I continue to use this terminology. And I think it is more natural to think of VA as derived from RVA, because RVA is something you can derive directly from the file, while VA can be obtained only when you already know the base address where you load the image.
Post 23 Aug 2015, 14:37
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
There are different useful applications for a data directive obeying the assumed address specification (including virtual at). I see the way it works now as introduction of another special case without any practical advantage.
Quote:
The very old PE specification that was the only one I used when implementing PE output into fasm back in 1999/2000

I didn't even know how to use a PC that time. But you are right regarding the "remnants". Though I think the wording and the value of a correct specification was just not well considered.

_________________
Faith is a superposition of knowledge and fallacy
Post 23 Aug 2015, 15:07
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7725
Location: Kraków, Poland
Tomasz Grysztar
I agree that a directive like "data fixups at base:size" would be more robust and perhaps even more fitting to fasm's paradigm of customizability. But the "data"/"end data" syntax plainly defines an area of file/image containing the given directory and allowing it to be affected by ORG would only cause confusion, it would look more like a bug rather than a feature.
Post 23 Aug 2015, 15:24
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
I agree that a directive like "data fixups at base:size" would be more robust

No, you're suggesting syntactical extensions, but my opinion is that data directive by itself just should obey the expected data/code loading address from the conceptual point of view.
Quote:
But the "data"/"end data" syntax plainly defines an area of file/image containing the given directory

I don't see any conceptual difference of this kind of definition versus any other kind of data or code definition. Any data definition directive is a plain definition of a file area.
Quote:
allowing it to be affected by ORG would only cause confusion

I experienced quite a lot of confusion when I saw that putting data/end data into a virtual at block had no effect on the data directory rva.
Quote:
it would look more like a bug rather than a feature

Yeah, considering all the things often reported as bugs, the rate of bug reports sounded like a weaker argument. Smile

_________________
Faith is a superposition of knowledge and fallacy
Post 23 Aug 2015, 15:50
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7725
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
I experienced quite a lot of confusion when I saw that putting data/end data into a virtual at block had no effect on the data directory rva.
Oh, that is perhaps an actual bug. DATA inside VIRTUAL should not be allowed.
Post 23 Aug 2015, 16:03
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
What? No! That's not what I meant.

Btw. I think you missed some of my arguments. E.g. there is no exact correspondence between RVAs and image file offsets. Multiple sections can overlap making the same region of an image file be mapped at multiple different RVAs. format PE does not allow overlapped sections, but still these do not violate the specification and are properly processed by the loader.

_________________
Faith is a superposition of knowledge and fallacy
Post 23 Aug 2015, 16:08
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7725
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
Btw. I think you missed some of my arguments. E.g. there is no exact correspondence between RVAs and image file offsets. Multiple sections can overlap making the same region of an image file be mapped at multiple different RVAs. format PE does not allow overlapped sections, but still these do not violate the specification and are properly processed by the loader.
Even that oldest specification draft that I used for initial implementation of "format PE" contains this paragraph:
Michael J. O'Leary in 'Portable Executable Format' wrote:
The virtual addresses for objects must be assigned by the linker such that they are in ascending order and adjacent, and must be a multiple of Object Align in the PE header.
(note that this old document used an "object" word for what we now call a "section").

And current specifications contain this old paragraph preserved, with updated terminology:
Microsoft in 'Microsoft Portable Executable and Common Object File Format Specification' wrote:
In an image file, the VAs for sections must be assigned by the linker so that they are in ascending order and adjacent, and they must be a multiple of the SectionAlignment value in the optional header.
Post 23 Aug 2015, 17:40
View user's profile Send private message Visit poster's website Reply with quote
l_inc



Joined: 23 Oct 2009
Posts: 881
l_inc
Tomasz Grysztar
Quote:
Even that oldest specification draft that I used for initial implementation of "format PE" contains this paragraph

The specification talks about virtual addresses / RVAs. The loader btw. does explicitly check and disallow non-contiguous or non-adjacent in-memory layout. The file offsets in contrast are allowed to overlap. This shows again that RVA is a really-really purely in-memory concept, and there's absolutely no single neither conceptual nor pragmatical reason to bind it to the locations on the hard drive.

_________________
Faith is a superposition of knowledge and fallacy
Post 23 Aug 2015, 18:07
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7725
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
The specification talks about virtual addresses / RVAs. The loader btw. does explicitly check and disallow non-contiguous or non-adjacent in-memory layout. The file offsets in contrast are allowed to overlap. This shows again that RVA is a really-really purely in-memory concept, and there's absolutely no single neither conceptual nor pragmatical reason to bind it to the locations on the hard drive.
My understanding of the specification is that such freedom is also not granted, though the fragment mentioning it is perhaps not unequivocal enough:
Microsoft in 'Microsoft Portable Executable and Common Object File Format Specification' wrote:
In an image file, the section data must be aligned on a boundary as specified by the FileAlignment field in the optional header. Section data must appear in order of the RVA values for the corresponding sections (as do the individual section headers in the section table).
Still, even if it is possible to have PE files following different rules, fasm's "format PE" implementation is such that you define image file and memory image side by side.
Post 23 Aug 2015, 19:36
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 7725
Location: Kraków, Poland
Tomasz Grysztar
l_inc wrote:
Quote:
The very old PE specification that was the only one I used when implementing PE output into fasm back in 1999/2000

I didn't even know how to use a PC that time.
As a funny sidenote: I can actually say the same about the time when that original PE specification was written. The copy of PE.TXT I had was dated 1993.
Post 23 Aug 2015, 19:39
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on YouTube, Twitter.

Website powered by rwasa.