flat assembler
Message board for the users of flat assembler.
 Home   FAQ   Search   Register 
 Profile   Log in to check your private messages   Log in 
flat assembler > Main > serializing pointers

Author
Thread Post new topic Reply to topic
vivik



Joined: 29 Oct 2016
Posts: 115
serializing pointers
Imagine that you need to save in a file some struct, which points to another struct. How would you do that?
Post 27 Mar 2017, 07:26
View user's profile Send private message Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 260
Location: Australia
I have devoted a considerable amount of time in the past to this/similar problems. See: https://2ton.com.au/library_as_html/mappedheap.inc.html by way of example.

Instead of pointers, all "memory" (e.g. file space) is referred to by offset instead of by physical memory location, solving such a use case.

I have gone on to create many derivative structures from this same underlying idea: https://2ton.com.au/library_as_html/mtree.inc.html

Hope this helps stir some thought Smile
Post 27 Mar 2017, 07:39
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6435
Location: Kraków, Poland
The fasm's symbolic information file format is another example of approaching a similar problem. It contains "dumps" of many structures that are used by fasm internally at run-time, and when preparing these structures to be saved in file the addresses in memory need to be changed into relative offsets within the sections of file. Each part of file is prepared within memory as a contiguous block and all pointers inside various structures are converted into relative offsets relative to the start of a given block.
Post 27 Mar 2017, 07:46
View user's profile Send private message Visit poster's website Reply with quote
vivik



Joined: 29 Oct 2016
Posts: 115
Thank you for your awesome replies, took a bit of time to digest them. I have a new question.

So, if my memory blocks are all over the place (they have to be that way in my case, since they may change their size at any moment), then before serializing, I'll first need to put all structs I'll need in a single contiguous block of memory.

This requires some form of reflection, just to know the size of structures and arrays, and which parts of them are pointers.

I wonder how to better store this reflection info.

(It feels like I'll end up writing a memory allocator)
Post 27 Mar 2017, 09:41
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 14923
Location: 6EQUJ5
There is no need to move the blocks around within memory. Just assign the offsets in ascending order when serialising.
Post 27 Mar 2017, 09:44
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6435
Location: Kraków, Poland
If your blocks are all over the place, you can create some data structure to store the pairings of memory addresses (values of pointers) with offsets in file. If memory usage is not a concern, a binary trie might be a recommendable choice. It would have to be done in two passes - first one to reserve offsets in file for every structure that you need to save (and store these offsets as values for memory address keys in the aforementioned data structure), then in the second pass replace all the pointers within the structures with the offsets reserved earlier and then save structures at the corresponding positions in file.
Post 27 Mar 2017, 09:50
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 14923
Location: 6EQUJ5
It can also be done in a single pass with a recursive function. The storage requirements would also be smaller.
Post 27 Mar 2017, 09:55
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6435
Location: Kraków, Poland

revolution wrote:
It can also be done in a single pass with a recursive function. The storage requirements would also be smaller.

What if there are circular references?
Post 27 Mar 2017, 09:57
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 14923
Location: 6EQUJ5
Fair enough Embarassed

If you know that your structures don't have back-references or circular-references then ... it can also be done in a single pass with a recursive function.

Laughing
Post 27 Mar 2017, 10:24
View user's profile Send private message Visit poster's website Reply with quote
vivik



Joined: 29 Oct 2016
Posts: 115

revolution wrote:
There is no need to move the blocks around within memory. Just assign the offsets in ascending order when serialising.



I planned to use memory mapped files, and writing into them is moving memory into them. This is why I said that I need to move all memory blocks into one. Probably should have said "copy" instead.

Awesome replies, thank you!
Post 27 Mar 2017, 10:27
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >

Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2005 phpBB Group.

Main index   Download   Documentation   Examples   Message board
Copyright © 2004-2016, Tomasz Grysztar.