flat assembler
Message board for the users of flat assembler.

Index > Windows > api for autosync memory block among several processes

Author
Thread Post new topic Reply to topic
sleepsleep



Joined: 05 Oct 2006
Posts: 12740
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 0010456
sleepsleep 12 Jul 2012, 15:25
hi,
is there win api that performs such functions.

PC SERVER
- contained 1 big text / binary file (8GB)

PC CLIENT 1
- network map the 1 big file in PC SERVER

PC CLIENT 2
- network map the 1 big file in PC SERVER

PC CLIENT 3
- network map the 1 big file in PC SERVER

whenever the 1 big file is edited or updated in PC SERVER, all PC CLIENT would automatically get updated too as if the 1 big file is loaded on every PC CLIENT (but the one big file must not get loaded on every PC CLIENT), it should only reference.

eg, PC CLIENT 1 request to read from memory 0040AAAA to 0040BBBB, then the window does the magic by itself and return what we requested.
Post 12 Jul 2012, 15:25
View user's profile Send private message Reply with quote
LostCoder



Joined: 07 Mar 2012
Posts: 22
LostCoder 12 Jul 2012, 18:47
You can try network file sharing with MapViewOfFile, however I'm not sure that MapViewOfFile will work with shared files.
Post 12 Jul 2012, 18:47
View user's profile Send private message Reply with quote
typedef



Joined: 25 Jul 2010
Posts: 2909
Location: 0x77760000
typedef 12 Jul 2012, 22:19
Just go with Winsock or Wininet (ftp, use APIs like InternetOpenFile, InternetSetFilePointer, InternetReadFile)

Make the server run on FTP with just simple commands like STAT,LIST,GET.
Post 12 Jul 2012, 22:19
View user's profile Send private message Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3175
Location: Denmark
f0dder 22 Jul 2012, 00:24
There's no built-in API to do this, and it's a somewhat complex task to do if you want to do it right (i.e., a robust implementation that's also fast and scales well).

Memory mapped files are not the way to go, at least not without an additional layer on top - quoting MSDN:
Quote:
With one important exception, file views derived from any file mapping object that is backed by the same file are coherent or identical at a specific time. Coherency is guaranteed for views within a process and for views that are mapped by different processes.

The exception is related to remote files. Although CreateFileMapping works with remote files, it does not keep them coherent. For example, if two computers both map a file as writable, and both change the same page, each computer only sees its own writes to the page. When the data gets updated on the disk, it is not merged.
Post 22 Jul 2012, 00:24
View user's profile Send private message Visit poster's website Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 12740
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 0010456
sleepsleep 25 Jul 2012, 15:35
i was thinking something like below, would it be something breakthrough?

eg,
a http webserver that more like asm scripting engine.
where the "programming" is done through compiled language, and the webserver call user proc via dll.
like a bridge for web application into windows api

m1 = memory place name
http://localhost/memalloc,1024,m1
reply ok

http://localhost/memput,m1,0,857
reply ok
// we put 857 in memory space [0]

user1 = http://localhost/memrun,adding,[0],3
reply 860
user2 = http://localhost/memrun,adding,[0],4
reply 861

http://localhost/memrun,adding,5,10
reply 15

http://localhost/memput,m1,5,this is a quick brown fox


proc adding a b
mov eax, dowrd [a]
add eax, dword [b]
endp
Post 25 Jul 2012, 15:35
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20300
Location: In your JS exploiting you and your system
revolution 25 Jul 2012, 15:41
This doesn't appear to be any sort of "breakthrough" as you put it. It is just like the standard DB access that every DB engine offers.

BTW: It is also really really slow.
Post 25 Jul 2012, 15:41
View user's profile Send private message Visit poster's website Reply with quote
hopcode



Joined: 04 Mar 2008
Posts: 563
Location: Germany
hopcode 25 Jul 2012, 15:42
sleepsleep wrote:
a http webserver that more like asm scripting engine
+1 from me. super cool idea Idea
fasm compiler online with feedback on local procs something using RPC (remote procedure call) and revisiting
the protocol if necessary.
Cheers,

_________________
⠓⠕⠏⠉⠕⠙⠑
Post 25 Jul 2012, 15:42
View user's profile Send private message Visit poster's website Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 12740
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 0010456
sleepsleep 25 Jul 2012, 22:10
revolution wrote:

This doesn't appear to be any sort of "breakthrough" as you put it. It is just like the standard DB access that every DB engine offers.

BTW: It is also really really slow.

thanks, as you said, it could be really slow, and tons of security problems, i will try brainstorm more later.

hopcode wrote:

+1 from me. super cool idea Idea
fasm compiler online with feedback on local procs something using RPC (remote procedure call) and revisiting
the protocol if necessary.
Cheers,

thanks too
Post 25 Jul 2012, 22:10
View user's profile Send private message Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3175
Location: Denmark
f0dder 26 Jul 2012, 13:49
sleepsleep: if you go the HTTP way, read up on the REST principles - you don't want to GET a 'memalloc' resource, you want to PUT or POST a 'memory' resource...

But as revolution wrote, HTTP is probably too much overhead for a shared memory protocol... especially if you need security, because you really need SSL then.
Post 26 Jul 2012, 13:49
View user's profile Send private message Visit poster's website Reply with quote
r22



Joined: 27 Dec 2004
Posts: 805
r22 26 Jul 2012, 18:52
Google's RESTful gdata api and Microsoft's WebDAV work fairly well with regards to security and efficiency. If by overhead you were criticizing the time it would take to fully implement in ASM then I'd agree, implementing a fully functional HTTP(S) server in ASM would be substantial overhead.

The original query doesn't really mention any security constraints. A simple TCP/IP Server/Client with the main file memory mapped on the server and temporary files memory mapped on each client would likely be the easiest most efficient solution.

The server code would be as follows (pseudo code)
Code:
// server main loop
WHILE Server_Is_Running
    FOR EACH Client IN Connected_Clients
        // did the client send a READ/WRITE command?
        Read = recv (Client, Cmd_Buffer, CMD_SIZE)
        IF Read == CMD_SIZE THEN
            IF Cmd_Buffer == CMD_READ THEN
                // get the parameters (offset) and (size)
                Read0 = recv (Client, Param_Offset_Buffer, PARAM_SIZE)
                Read1 = recv (Client, Param_Size_Buffer, PARAM_SIZE)
                IF Read0 == PARAM_SIZE && Read1 == PARAM_SIZE THEN
                    // give the client the requested portion of memory
                    send (Client, Cmd_Buffer, CMD_SIZE)
                    send (Client, Param_Offset_Buffer, PARAM_SIZE)
                    send (Client, Param_Size_Buffer, PARAM_SIZE)
                    send (Client, Mapped_File_View_Ptr + Param_Offset_Buffer, Param_Size_Buffer)
                END IF
            ELSE IF Cmd_Buffer == CMD_WRITE THEN
                // get the parameters (offset) and (size)
                Read0 = recv (Client, Param_Offset_Buffer, PARAM_SIZE)
                Read1 = recv (Client, Param_Size_Buffer, PARAM_SIZE)
                IF Read0 == PARAM_SIZE && Read1 == PARAM_SIZE THEN
                    // get the parameter (data) to write
                    Read = recv (Client, Data_Buffer, Param_Size_Buffer)
                    IF Read == Param_Size_Buffer THEN
                        // update the memory
                        CopyMemory (Mapped_File_view_Ptr + Param_Offset_Buffer, Data_Buffer, Param_Size_Buffer)
                        // Tell all clients of the updated portion of the memory
                        FOR EACH c IN Connected_Clients
                            send (c, Cmd_Buffer, CMD_SIZE)
                            send (c, Param_Offset_Buffer, PARAM_SIZE)
                            send (c, Param_Size_Buffer, PARAM_SIZE)
                            // choose whether you want clients to receive updated data automatically
                            // or if they should just be notified and have to request the data
                            send (c, Data_Buffer, Param_Size_Buffer)
                        END FOR
                    END IF
                END IF
            END IF
        END IF
    END FOR
END WHILE
    

In the above case the server is processing synchronously to simply the pseudo code.

The simple protocol is fairly easy to implement.
Code:
{Read|Write}{Offset}{Size}{Data}
    


You could optimize the protocol further by just having
Code:
{Offset} {Size|-Size} {Data}
    

Where the -negative size would denote a Write.
Or use a separate tcp port for Read and Write requests.
Or a command and data tcp port.

Anyways, there are infinitely many ways to program the skinning of a cat.
Post 26 Jul 2012, 18:52
View user's profile Send private message AIM Address Yahoo Messenger Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.