flat assembler
Message board for the users of flat assembler.

Index > Main > Help with choosing a challenging project

Author
Thread Post new topic Reply to topic
Inagawa



Joined: 24 Mar 2012
Posts: 153
Inagawa 09 Jun 2012, 18:02
As (typedef I think it was) said a very long time, I should ask for a project after I learn a few things.

Well, I think it's that time. I mean, I made the PerformanceCounter macro, but I'm kinda lost now. I have no project and the only thing that is on my mind (making a memory allocator) is far too advanced for me.

So I'm asking, is there anyone out here who would have something challenging (but not impossible), that would make me learn new things? Force me to think more in assembly? I'd be grateful if someone took the time to think up a challenge. My skill is still weak, so take it easy on me, please. Smile
Post 09 Jun 2012, 18:02
View user's profile Send private message Reply with quote
mindcooler



Joined: 01 Dec 2009
Posts: 423
Location: Västerås, Sweden
mindcooler 09 Jun 2012, 19:28
A memory allocator is only as advanced as you make it. You can make a simple heap with only tens of lines of code.
Post 09 Jun 2012, 19:28
View user's profile Send private message Visit poster's website MSN Messenger ICQ Number Reply with quote
Inagawa



Joined: 24 Mar 2012
Posts: 153
Inagawa 10 Jun 2012, 06:45
Well the trouble is, where to begin? How do I actually allocate memory? Some people say you do it by syscall, but others say you do it by HeapAlloc and should never touch syscall unless you're writing an OS or driver. I have attempted to dissect code of two memory allocators in C and for the life of me I can't find the part that actually does the allocation
Post 10 Jun 2012, 06:45
View user's profile Send private message Reply with quote
LostCoder



Joined: 07 Mar 2012
Posts: 22
LostCoder 10 Jun 2012, 07:56
In Windows the fastest thing I know is VirtualAlloc, VirtualFree function.
Post 10 Jun 2012, 07:56
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20306
Location: In your JS exploiting you and your system
revolution 10 Jun 2012, 08:23
Inagawa wrote:
Well the trouble is, where to begin? How do I actually allocate memory? Some people say you do it by syscall, but others say you do it by HeapAlloc and should never touch syscall unless you're writing an OS or driver. I have attempted to dissect code of two memory allocators in C and for the life of me I can't find the part that actually does the allocation
You appear to be mixing Linux syscalls with Windows APIs there. If you want to use the C HLL library functions for memory allocation then you have somewhat made the whole discussion moot. For assembly you would most likely want to use the OS provided functions (that is what C uses internally of course) to allocate memory for your app. There really isn't any other way to do it since the OS will have control over what you can and can't do.
LostCoder wrote:
In Windows the fastest thing I know is VirtualAlloc, VirtualFree function.
It is not about fastest here. The Virtual* functions have a particular purpose where their use makes sense. And the Heap* functions have a different purpose where their use makes sense. The Local* functions have their particular purpose also. You have to look at what your app needs to decide what is the right set of functions to use.
Post 10 Jun 2012, 08:23
View user's profile Send private message Visit poster's website Reply with quote
Inagawa



Joined: 24 Mar 2012
Posts: 153
Inagawa 10 Jun 2012, 08:51
I see. I'll try to create something and then ask some specific questions instead.
Post 10 Jun 2012, 08:51
View user's profile Send private message Reply with quote
AsmGuru62



Joined: 28 Jan 2004
Posts: 1619
Location: Toronto, Canada
AsmGuru62 10 Jun 2012, 11:52
I coded a few heaps for Windows, so I can provide some advice.
Basically, you use Win32 VirtualAlloc to allocate large segments of memory (say, 8Mb per segment).
Then to allocate small blocks (like 22 bytes for some string or a small structure) you cut the piece (22 byte piece) of that big block and return its address to the caller. Once the segment is ended -- call VirtualAlloc to get a fresh one.
When memory is freed -- try to see if the released block has free neighbouring block(s) and make a larger block from these.

The rest is optimization. Always keep the pointer to next free block, so
you perform less searching for a block.

Write a function to defragment all segments in heap.

And of course, diagnostics are very important:
- detect memory leaks
- detect when a block passed to free()/realloc() is invalid
- detect the memory block overrun

Maybe, when creating a heap object -- use some parameter to switch
diagnostics ON/OFF, because diagnostics will slow down your code considerably.

The memory block structure in memory can look like that:
Code:
+----------+------------------------+----------+
| BLK_HEAD |  Block data bytes ...  | BLK_TAIL |
+----------+------------------------+----------+
           |
           |<-- This address returned to the caller
    

These structures are following each other within a segment.
BLK_HEAD is a structure, containing the size of a block, flags (like is the block free or allocated) and in case of diagnostics -- some tag, which indicates that this is indeed the valid block.

BLK_TAIL is optional and usually (in case diagnostics are ON) contains the bytes to detect if a block has been damaged or overwritten.

The memory returned from your allocator should be aligned to 8 bytes to comply with the data alignment requirements. It may not, but then most of CPUs will slow down the code, which works with allocated data.

Example, the block of 22 bytes must be allocated.
It means the whole size of the area is: BLK_HEAD + 22 + BLK_TAIL.

BLK_HEAD is usually 8 bytes, so working area is also aligned.
So, you have now 30 bytes + BLK_TAIL.

Closest 8 byte aligned size to this is 32 bytes (8*4), so
your BLK_TAIL is just 2 bytes, but for diagnostics you must fill it
with some value, like 0CDh and when block is getting freed -- you
check if the byte following the 22 bytes is still 0CDh and if it is not - block has been corrupted.

Isn't it fun?!..


Last edited by AsmGuru62 on 10 Jun 2012, 12:00; edited 1 time in total
Post 10 Jun 2012, 11:52
View user's profile Send private message Send e-mail Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20306
Location: In your JS exploiting you and your system
revolution 10 Jun 2012, 12:00
AsmGuru62: That is basically the operation of HeapAlloc and LocalAlloc. You do realise that you are reinventing the wheel there, right? Wink
Post 10 Jun 2012, 12:00
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 10 Jun 2012, 12:13
Well, I need OS independent heap manager for FreshLib and probably will make one someday.
Isn't it better to keep some memory map for allocated/free blocks at some different place. For example, bit array with 1bit per 16bytes of memory.
This way, it is less probable, that the user can destroy the heap by writing outside the allocated block. You can separate the heap and the memory map by guard page.
Post 10 Jun 2012, 12:13
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
AsmGuru62



Joined: 28 Jan 2004
Posts: 1619
Location: Toronto, Canada
AsmGuru62 10 Jun 2012, 12:41
Yes. The custom heap, however, provides more flexibility in some cases.
Or it can be optimized to a higher degree, since the owner of code is you and not Microsoft.
Post 10 Jun 2012, 12:41
View user's profile Send private message Send e-mail Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.