flat assembler
Message board for the users of flat assembler.

flat assembler > Compiler Internals > fasm as DLL

Goto page Previous  1, 2, 3, 4
Author
Thread Post new topic Reply to topic
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7190
Location: Kraków, Poland
sesey wrote:
How can I compile this source code as x64.dll?
This is a bit complex, it requires the same macros as Linux x64 version and rewrite of OS abstraction layer, I may take a look at this in the future.
Post 21 Aug 2018, 20:15
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7190
Location: Kraków, Poland
ProMiNick wrote:
But internal variable __FILE__ without value can produce bugs in virtual blocks (when they are as additional files).
Do not worry, the engine has it all figured out, you just need to be careful to use its functions properly (I did my best to well-document their interfaces).

As a quick demonstration I made a simplest DLL version of fasmg. It just exposes the main interfaces of engine (and uses hard-coded values for most of the settings). It operates on local variables, so should be thread-safe.

The attached files have to be in the source/windows/dll directory in fasmg source tree to assemble correctly. It can only be self-hosted with fasmg. Note that selfhost.inc has additional macros to convert MOV to LEA when EBP-relative addresses need to be put into a register.

The main function is:
fasmg_Assemble(source_string, source_path, reserved, output_path, stdout, stderr)
All arguments are optional, every one of them is allowed to be NULL.
  • source_string is the text to be assembled directly from memory. Normal versions of fasmg use this for the instructions added from command line with -I switch, but this can just as well be used to assemble complete source not stored in any file.
  • source_path is the path to source file. If both in-memory source and source file are provided, their concatenation is assembled.
  • reserved for a parameter that might be used to receive output in memory. Currently not implemented.
  • output_path is the path to file for the output. When not provided, the output path is generated from the input path (just like the command-line fasmg does it). When there is neither path provided, output is not written anywhere.
  • stdout should be a handle to file or pipe that receives the messages generated with DISPLAY command.
  • stderr should be a handle to file or pipe that receives the error messages.

Function returns the number of errors encountered in the assembly, so 0 indicates success. When there were errors, details have been written into stderr (if provided).
Negative values returned indicate fatal errors: -1 when assembler ran out of memory, -2 when maximum number of passes has been reached without a resolved assembly, -3 when writing output file has failed for some reason.

I kept this version as simple as possible, mainly to demonstrate how the bare engine can be used. If you wanted a pure memory-to-memory assembly, you'd need to add some special handling. You'd have to allocate a big enough memory block (get_output_length function can give the required size) and then perhaps use read_from_output to copy data.

EDIT: Attachment removed - see below.


Last edited by Tomasz Grysztar on 13 Sep 2018, 07:57; edited 3 times in total
Post 21 Aug 2018, 20:51
View user's profile Send private message Visit poster's website Reply with quote
sesey



Joined: 21 Aug 2018
Posts: 3
Tomasz Grysztar wrote:
sesey wrote:
How can I compile this source code as x64.dll?
This is a bit complex, it requires the same macros as Linux x64 version and rewrite of OS abstraction layer, I may take a look at this in the future.


I thank you for your help. I will wait for your future work.
Post 22 Aug 2018, 09:17
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7190
Location: Kraków, Poland
I have added a simple modification to the above FASMG.DLL to allow a pure memory-to-memory assembly.

The reserved argument became output_region which should either be a NULL (when no in-memory output is needed) or a pointer to this simple structure:
Code:
struct MEMORY_REGION
        address dd ?
        size dd ?
ends    
It should be initially zeroed or may be filled with information about a block of memory pre-allocated with VirtualAlloc. When the assembly is successful, the structure is filled with information about the size of generated output and the address where it is stored. If the structure provided a memory region but it was too small to hold the output, a region is re-allocated with VirtualAlloc, releasing the previous one with VirtualFree. This way you can keep calling fasmg_Assemble and have it re-use the same region of memory for output. You can also initialize the address field to point to a region prepared with VirtualAlloc with MEM_RESERVE flag and set the initial size to zero, fasmg_Assemble should then be able to keep using the reserved address, committing as much memory as needed for the generated output.

For pure memory-to-memory assembly both source_path and output_path should be NULL.

EDIT: Attachment removed - see below.


Last edited by Tomasz Grysztar on 13 Sep 2018, 07:57; edited 1 time in total
Post 23 Aug 2018, 17:25
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7190
Location: Kraków, Poland
I have now included the sources of this simple DLL in the main repository of fasmg: https://flatassembler.net/fossil/repo/fasmg/info/3d9126369323f2c7

EDIT: the sources are not also included in the basic package. The added files are small and I like them enough to show them in the official distribution as a demonstration how fasmg core can be used in multi-threaded environment.
Post 31 Aug 2018, 15:30
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7190
Location: Kraków, Poland
Tomasz Grysztar wrote:
sesey wrote:
How can I compile this source code as x64.dll?
This is a bit complex, it requires the same macros as Linux x64 version and rewrite of OS abstraction layer, I may take a look at this in the future.
I have made an experimental FASMX64.DLL that uses the same macros as Linux x64 version. It seems to work, but keep in mind that the limitation of this hack is that it is only going to work when all the memory blocks fit in the low 4G addressing space. When you allocate memory for fasm you need to make sure that allocation is in the low memory, also when you provide source text in memory, you need to ensure that is fits in low 4G.

At the same time I also added a mutex in FASM.DLL. I think I have been making a bad example by not using something like that when the core is obviously and dangerously not thread-safe.

I believe it might also be possible to make FASMX64.DLL not require mutex and just use a per-thread memory block for all variables by addressing it with a base register like R14 (similarly to how FASMG.DLL addresses everything relative to EBP). However this would require some careful setup, to allocate locals block in low 4G space for every function call. For now I used a mutex there, too.
Post 13 Sep 2018, 09:34
View user's profile Send private message Visit poster's website Reply with quote
bazizmix



Joined: 15 Jan 2016
Posts: 42
So. I made a 64-bit fasm. Exe and dll. Tested only for PE and PE64.
It can be compiled as 32-bit from 64-bit and versa.


Description: Source and example
Download
Filename: fasmdll64.7z
Filesize: 230.33 KB
Downloaded: 106 Time(s)

Post 02 Nov 2018, 14:45
View user's profile Send private message Reply with quote
ProMiNick



Joined: 24 Mar 2012
Posts: 259
Location: Russian Federation, Sochi
Because of my attachments limit exceded - https://yadi.sk/d/OED0phH1DFaeMg
There are located use of fasm.dll previously posted above, and added use of fasmarm.dll (ofcource with fasmarm.dll inside)

P.S. sorry guys - no sources for fasmarm.dll. Source looks ugly - I adapt them just to get working out.
For normal sources to fasmarm.dll ask revolution.
Post 18 Nov 2018, 19:12
View user's profile Send private message Send e-mail Reply with quote
Sewer56



Joined: 23 Dec 2018
Posts: 2
Quite recently, I started work on my own minimal C#/.NET wrapper to try and leverage the recently released "FASMX64" so I could make 64-bit development with FASM a possibility in the high-level world.

I have had some serious problems getting FASMX64 (from the first post/1.73 core) to run myself; no matter what I did - the .NET Core / .NET Framework process would crash the moment FASMX64.dll was loaded.

I have tried the following methods of loading the library into the process:
1. P/Invoke (uses LoadLibrary under the hood).
2. LoadLibraryA/W
3. Dll Injection
4. Manual Mapping DLL Injection (using an external library)

To no avail. Immediate crash with every attempt after ~4-6 hours of attempts.

Seeing that in the post above, bazizmix also posted his attempt at experimental X64 FASM, I tried his solution and it fortunately loaded without problems. It worked for fasm_GetVersion, but fasm_Assemble was broken 100% of the time for assembling ASM with errors and Random% of the time otherwise.

-----------------------------------------

The good news at least is that in the end, after 2-3 hours of debugging with Cheat Engine and the source open beside in Notepad++, I did manage to make things work.

I took the modified source uploaded by bazizmix and have made the following changes/fixes:

1. I changed how hDisplayPipe is retrieved for x64 from the function parameters as the stack offset used was incorrect and pointed into the "shadow space" of the Microsoft X64 calling convention.

This caused random crashes as FASMDLL would sometimes try to read uninitialized memory when trying to obtain the display pipe handle because the contents of the "shadow space" are undefined (it is not zero filled and typically contains old stack data),

2. The esp_save variable was marked as a qword but when it was backed up, the value 32-bit register was stored as opposed to the full length 64-bit register. This led to issues if the stack was in the 64-bit address range.

3. In assemble.inc, the error_handler which jumps the assembler to the error table was not using the full width registers for esp, eax. This became an issue when adding a negative value was involved, as the high bits of the register were not set - only half a register. I changed them to the _esp, _eax macros respectively and confirmed the solution still compiles and works for x86 as well as x64.

This led to crashes 100% of the time there was an error because of a bad jump to uninitialized/random memory.

All changes are commented with [CHANGE] in the source files fasmdll.asm and assemble.inc where they are applied. I also left some general comments

My modified version is linked in the attachments.
As for the .NET wrapper library, you can find it here: https://github.com/Reloaded-Project/Reloaded.Assembler

Merry Christmas everyone!

PS.
My solution still adheres to the original given limitations of FASMX64, I did not modify the actual assembler itself as that would be a rather large undertaking. This means that 64bit pointers for lpSource, lpMemory are a no-go; they must be addressed in the first 4G of Memory (32bit pointers).


Description:
Download
Filename: fasmdll64mod.7z
Filesize: 221.45 KB
Downloaded: 50 Time(s)

Post 23 Dec 2018, 18:11
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7190
Location: Kraków, Poland
Unless DLL ends up loaded at high (above 4G) base address, it should not be needed to modify the core fasm (like "error_handler" routine). But perhaps bazimix's version does not use the same tricks - my macros (originally written to make the x64 Linux executable) redefine ESP to always mean RSP, for example.

Interestingly, even though your notes concern bazimix's version, my version also had a bug in how the value of hDisplayPipe was retrieved - I have corrected it now (the attachment in first post is updated).

As for "esp_save", my version does not have it, the value of RSP is preserved in R12 (not used by fasm's core).
Post 23 Dec 2018, 18:58
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7190
Location: Kraków, Poland
Please try the updated version of my (official) adaptation. I have noticed and corrected a bug in "invoke" macro, perhaps this is what was causing crash on load (DLLEntryPoint in my version calls CreateMutex with "invoke").
Post 23 Dec 2018, 19:37
View user's profile Send private message Visit poster's website Reply with quote
Sewer56



Joined: 23 Dec 2018
Posts: 2
I've tested the updated official version with 50 runs of the assembler by hand and can confirm that everything works perfectly now. Very Happy
Post 23 Dec 2018, 20:26
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2, 3, 4

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2019, Tomasz Grysztar.

Powered by rwasa.