suppose we have a dll for 64 bit windows such as


Now, the dll is loaded at different addresses each time that is is loaded, so the actual addresses of f and g are not fixed. However, I looked at the instruction mov rax,g in a debugger, and it indeed points to the correct function g every time. Then, I look at the instruction mov rax,g in a raw hex editor at it was set to address 0x00403ADE. What is going on? How does fasm tell windows that it needs to fix(rewrite) something as complicated as this every times the dll loads? To what extent can we expected things to be fixed correctly after dll load?

For a DLL a relocation section should be included, which specifies where address fixups should take place. The assembler keeps track of which addresses and sizes that need updating. So, that when it creates the fixup section the correct data is stored. Then the loader reads the fixup data, and makes the required changes.

Yeah - I guess that must be the way.
Challenge: Can you make a 64 bit dll in fasm that works when not relocated, but breaks when relocated by windows?

Seriously, though, has the relocator mechanism been proven correct for all files (of lets say size < 4MB)?

I have a number of very radical PE64 files and come across unusual errors [1]. Most tools claim they are not even executables, but Windows executes them. RIP relative addressing could be used to eliminate relocation requirement.

https://code.google.com/p/corkami/wiki/PE

...might give you some ideas for avenues of extended research. corkami has compiled a nice variety of 32-bit PE files for testing requirements of loader across windows versions. The 64-bit loader has just as much flexibility it seems.