flat assembler
Message board for the users of flat assembler.

Index > Windows > Is 0ffffffffh (4GB-1) mem max in Fasm32?

Goto page Previous  1, 2
Author
Thread Post new topic Reply to topic
Overclick



Joined: 11 Jul 2020
Posts: 669
Location: Ukraine
Overclick 20 Jun 2022, 21:13
Seems you never used SWAP. I monitor my RAM and it have to be a lot of apps running at same time or keeped minimized to begin using SWAP.
It is absolutely uncommon situation where some app still eating free memory.
Post 20 Jun 2022, 21:13
View user's profile Send private message Visit poster's website Reply with quote
Fastestcodes



Joined: 13 Jun 2022
Posts: 75
Fastestcodes 21 Jun 2022, 04:36
32bit sys, 4GB-1 Fasm mem, 4GB ramdrive Z:, 4GB ramdrive Y:,4GB ramdrive X: ?
Post 21 Jun 2022, 04:36
View user's profile Send private message Reply with quote
Fastestcodes



Joined: 13 Jun 2022
Posts: 75
Fastestcodes 21 Jun 2022, 04:41
mem1[0ffffffffh]
mem2[0ffffffffh]
mem3[0ffffffffh]...

64GB pendrive works on 32bit sys.
Post 21 Jun 2022, 04:41
View user's profile Send private message Reply with quote
I



Joined: 19 May 2022
Posts: 58
I 21 Jun 2022, 09:39
[quote="DimonSoft"]
revolution wrote:
And even if the rogue program touches its pages all the time, it is easy to find by watching running which program causes everything to slow down.
No need to keep touching memory, use VirtualLock instead Twisted Evil

I'll add my 2c on pagefile FWIW, (not much probably). There use to be a time the pagefile and swapping out was evil incarnate sometimes leaving the PC unresponsive which in turn lead to using the power button and that in turn often resulted in disk corruption and data loss so it was a sigh of relief to have it disabled. Haven't seen that for a long time now and use the pagefile because virtual memory size is a combination of pagefile size and RAM size. Without it one can run out of virtual memory before RAM leaving multiple gigabytes of RAM unusable! Swapping out to pagefile was for modifiable data, read only data can still be evicted without use of a pagefile because it resides on the disk it was loaded from, such as executables!

Personally I never liked MS terminology with virtual memory, seem to confuse rather than be intuitive.

Below is an example of using nearly 4GB of GPU VRAM (cudaMalloc) and 3584MiB pagefile.


You can see Total Page/Commit - RAM = 28080 - 24496 = 3584, the size of the pagefile. Before allocating VRAM available Page/Commit is 20966, afterward it drops to 17522, 20966 - 17522 = 3444MiB, RAM has only dropped 73MiB and Pagefile.sys usage hasn't noticeably changed, still at 0.179% (>7MiB).

Note that what they call pagefile (MEMORYSTATUSEX) and commit (PERFORMANCE_INFORMATION) appear to be the same, aka virtual memory, in particular that which can be physically addressed, not the maximum the CPU can address. For example the CPU may be able to address thousands of gigabytes of RAM but if you have only 8GB installed that's all you actually get to use.

Different Windows versions may give different results but that's were I am for now.


Description: Before cudaMalloc
Filesize: 10.24 KB
Viewed: 3426 Time(s)

VM2.png


Description: After cudaMalloc
Filesize: 12.1 KB
Viewed: 3426 Time(s)

VM1.png


Post 21 Jun 2022, 09:39
View user's profile Send private message Reply with quote
Furs



Joined: 04 Mar 2016
Posts: 2523
Furs 21 Jun 2022, 13:00
It's worth to note that on Windows, there is no overcommit. This can be both a good and bad thing. So in this case a page file "helps" even if it's never used because it gives more room for overcommits.

Overcommit is when an app commits memory (not just reserves address space for it, but commits it), even though it doesn't touch it. The memory itself doesn't really get physically allocated until it is actually touched. This happens on both Windows and Linux.

The difference is that on Windows, if you have no more "free memory" available (which is your RAM + page file, not just RAM), it will fail to commit in the first place, even if it's not used later. Linux allows overcommit so this is not a benefit of swap there (although it can be adjusted or turned off, obviously).

But overcommit does have its own issues of course; if the app actually uses the memory it will obviously crash (the OOM killer will kick in). Anyway, Windows benefits from swap more than Linux due to this.
Post 21 Jun 2022, 13:00
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20363
Location: In your JS exploiting you and your system
revolution 21 Jun 2022, 13:41
fasm for Windows console is a greedy app. It allocates all RAM first before it knows what it needs.

I posted a patch to change that behaviour by reserving the address space, and only committing when there was an access exception. It worked fine, and there was no need to use the -m switch unless one wanted to forcibly limit the maximum usage. And it could take advantage of the 1GB in the upper half of the address space, for those really large assembly tasks that can't fit into 2GB.

Apps can be fixed to not be greedy. But many apps don't do it.

Some of the worst culprits can be the GC based languages. Some just seem to keep allocating until they get refused and only then run GC. Fuck them. Smile Clean up your crap and don't assume you are the only app running and can use all the RAM.
Post 21 Jun 2022, 13:41
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20363
Location: In your JS exploiting you and your system
revolution 21 Jun 2022, 17:42
I found the old post making Windows fasm be friendly with RAM and use the upper memory section also if you boot with /3GB.

https://board.flatassembler.net/topic.php?t=17667
Post 21 Jun 2022, 17:42
View user's profile Send private message Visit poster's website Reply with quote
Fastestcodes



Joined: 13 Jun 2022
Posts: 75
Fastestcodes 22 Jun 2022, 03:55
32bit fasmw exe on 64bit win,GlobalAlloc:
0ffffffff Byte(4GB-1): outfile 0
2GB: outfile 0
1GB: works, outfile good.
Post 22 Jun 2022, 03:55
View user's profile Send private message Reply with quote
I



Joined: 19 May 2022
Posts: 58
I 22 Jun 2022, 04:45
Made some changes to 'Virtual Memory (MiB)', now with pagefile and commit named Virtual as in virtual addressing. Hopefully not too many mistakes.
Code:
format PE64 GUI 4.0
entry start

include 'win64a.inc'

struct  MEMORYSTATUSEX
 dwLength                   dd ?
 dwMemoryLoad               dd ?
 ullTotalPhys               dq ?
 ullAvailPhys               dq ?
 ullTotalPageFile           dq ?
 ullAvailPageFile           dq ?
 ullTotalVirtual            dq ?
 ullAvailVirtual            dq ?
 ullAvailExtendedVirtual    dq ?
ends


struct  ENUM_PAGE_FILE_INFORMATION
 cb                             dd ?
 Reserved                       dd ?
 TotalSize                      dq ?
 TotalInUse                     dq ?
 PeakUsage                      dq ?
ends

struc rt L,T,R,B {
    .L  dd  L
    .T  dd  T
    .R  dd  R
    .B  dd  B
}


;----------------------------------------
section '.text' code readable executable

 _title         db 'Virtual Memory (MiB)',0        ; Window Title
 _class         db 'Low',0
 wsh2a          db '%I64u',0
 _T1            db 'Total',0
 _T2            db 'Used',0
 _T3            db 'Available',0
 _T4            db 'RAM',0
 _T5            db 'Virtual',0
 _T6            db 'PageFile',0

 start:
 frame
        sub     rsp,8*11

        xor     ecx,ecx
        call    [GetModuleHandle]
        mov     [wc.hInstance],rax
        xor     ecx,ecx
        mov     edx,IDI_ASTERISK
        call    [LoadIcon]
        mov     [wc.hIcon],rax
        mov     [wc.hIconSm],rax
        xor     ecx,ecx
        mov     edx,IDC_ARROW
        call    [LoadCursor]
        mov     [wc.hCursor],rax
        lea     rcx,[wc]
        call    [RegisterClassEx]
        test    rax,rax
        jz      exit

        mov     [Gms.dwLength],sizeof.MEMORYSTATUSEX

        mov     [lfont.lfHeight],14
        mov     [lfont.lfWidth],0
        mov     [lfont.lfWeight],300
        mov     [lfont.lfQuality],DRAFT_QUALITY
        mov     dword[lfont.lfFaceName],'Aeri'
        mov     dword[lfont.lfFaceName+4],'al'
        mov     dword[lfont.lfFaceName+8],0
        lea     rcx,[lfont]
        call    [CreateFontIndirect]
        mov     [lfhandle],rax


        mov     ecx,WS_EX_TOPMOST
        lea     rdx,[_class]
        lea     r8,[_title]
        mov     r9d,WS_VISIBLE or WS_BORDER or WS_SYSMENU
        mov     qword[rsp+20h],10
        mov     qword[rsp+28h],10
        mov     qword[rsp+30h],254
        mov     qword[rsp+38h],95
        mov     qword[rsp+40h],0
        mov     qword[rsp+48h],0
        mov     rax,[wc.hInstance]
        mov     [rsp+50h],rax
        mov     qword[rsp+58h],0
        call    [CreateWindowEx]
        mov     [mainhwnd],rax

        call    TimerProc1
        mov     rcx,[mainhwnd]
        mov     edx,555
        mov     r8d,1000
        lea     r9,[TimerProc1]
        call    [SetTimer]

        mov     rcx,[mainhwnd]
        mov     edx,SW_SHOWNORMAL
        call    [ShowWindow]
        mov     rcx,[mainhwnd]
        call    [UpdateWindow]
 msg_loop:
        lea     rcx,[msg]
        xor     edx,edx
        xor     r8,r8
        xor     r9,r9
        call    [GetMessage]
        cmp     eax,1
        jb      end_loop
        jne     msg_loop
        lea     rcx,[msg]
        call    [TranslateMessage]
        lea     rcx,[msg]
        call    [DispatchMessage]
        jmp     msg_loop
 end_loop:

 Finito:
        lea     rcx,[mainhwnd]
        mov     rdx,555
        call    [KillTimer]
 exit:
        xor     ecx,ecx
        call    [ExitProcess]

;-------------------------------------------
 WindowProc:
       sub      rsp,8 * 11
       mov      [hwnd],rcx

        cmp     edx,WM_DESTROY
        je      wmdestroy

        cmp     edx,WM_PAINT
        je      PaintP

        cmp     edx,WM_ERASEBKGND
        je      wmErase

 defwndproc:
        call    [DefWindowProc]
        jmp     finish

 wmErase:
        mov     eax,1
        jmp     finish

 PaintP:
        mov     rcx,[hwnd]
        lea     rdx,[PaintS]
        call    [BeginPaint]
        mov     [hDC],rax
        mov     rcx,[hwnd]
        call    Painter
        mov     rcx,[hwnd]
        lea     rdx,[PaintS]
        call    [EndPaint]
        xor     eax,eax
        jmp     finish

 wmdestroy:
        xor     ecx,ecx
        call    [PostQuitMessage]
        xor     eax,eax

 finish:
        add   rsp,8 * 11
        ret

  Painter:
        sub     rsp,8 * 9

        lea     rdx,[Client]
        call    [GetClientRect]
        mov     rcx,[hDC]
        call    [CreateCompatibleDC]
        mov     [hdcMem],rax
        mov     eax,[Client.R]
        sub     eax,[Client.L]
        mov     [Width],rax
        mov     eax,[Client.B]
        sub     eax,[Client.T]
        mov     [Height],rax
        mov     rcx,[hDC]
        mov     rdx,[Width]
        mov     r8,[Height]
        call    [CreateCompatibleBitmap]
        mov     [hbmMem],rax
        mov     rcx,[hdcMem]
        mov     rdx,[hbmMem]
        call    [SelectObject]
        mov     [hbmOld],rax

        mov     rcx,[hdcMem]
        mov     rdx,[lfhandle]
        call    [SelectObject]
        mov     rcx,[hdcMem]
        mov     edx,TRANSPARENT
        call    [SetBkMode]
        mov     rcx,[hdcMem]
        lea     rdx,[Client]
        mov     r8d,COLOR_BTNFACE+1
        call    [FillRect]

        mov     rcx,[hdcMem]
        xor     edx,edx
        call    [SetTextColor]
        mov     rcx,[hdcMem]
        lea     rdx,[_T1]
        mov     r8,-1
        lea     r9,[T1]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]
        mov     rcx,[hdcMem]
        lea     rdx,[_T2]
        mov     r8,-1
        lea     r9,[T2]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]
        mov     rcx,[hdcMem]
        lea     rdx,[_T3]
        mov     r8,-1
        lea     r9,[T3]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]

        mov     rcx,[hdcMem]
        xor     edx,edx
        call    [SetTextColor]
        mov     rcx,[hdcMem]
        lea     rdx,[_T4]
        mov     r8,-1
        lea     r9,[T4]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]
        mov     rcx,[hdcMem]
        lea     rdx,[_T5]
        mov     r8,-1
        lea     r9,[T5]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]
        mov     rcx,[hdcMem]
        lea     rdx,[_T6]
        mov     r8,-1
        lea     r9,[T6]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]

        mov     rcx,[hdcMem]
        mov     edx,0f00000h
        call    [SetTextColor]

        mov     rcx,[hdcMem]
        lea     rdx,[_V1]
        mov     r8,-1
        lea     r9,[V1]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]
        mov     rcx,[hdcMem]
        lea     rdx,[_V2]
        mov     r8,-1
        lea     r9,[V2]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]
        mov     rcx,[hdcMem]
        lea     rdx,[_V3]
        mov     r8,-1
        lea     r9,[V3]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]

        mov     rcx,[hdcMem]
        lea     rdx,[_V4]
        mov     r8,-1
        lea     r9,[V4]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]
        mov     rcx,[hdcMem]
        lea     rdx,[_V5]
        mov     r8,-1
        lea     r9,[V5]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]
        mov     rax,[Gms.ullAvailPhys]
        cmp     rax,[Gms.ullAvailPageFile]
        jbe     @f
        mov     rcx,[hdcMem]
        mov     edx,0f0h
        call    [SetTextColor]
 @@:
        mov     rcx,[hdcMem]
        lea     rdx,[_V6]
        mov     r8,-1
        lea     r9,[V6]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]
        mov     rcx,[hdcMem]
        mov     edx,0f00000h
        call    [SetTextColor]

        mov     rcx,[hdcMem]
        lea     rdx,[_V7]
        mov     r8,-1
        lea     r9,[V7]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]
        mov     rcx,[hdcMem]
        lea     rdx,[_V8]
        mov     r8,-1
        lea     r9,[V8]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]
        mov     rcx,[hdcMem]
        lea     rdx,[_V9]
        mov     r8,-1
        lea     r9,[V9]
        mov     qword[rsp+20h],DT_RIGHT
        call    [DrawText]

        mov     rcx,[hDC]
        xor     edx,edx
        xor     r8d,r8d
        mov     r9d,[Client.R]
        mov     eax,[Client.B]
        mov     [rsp+20h],rax
        mov     rax,[hdcMem]
        mov     [rsp+28h],rax
        mov     qword[rsp+30h],0
        mov     qword[rsp+38h],0
        mov     qword[rsp+40h],SRCCOPY
        call    [BitBlt]
        mov     rcx,[hdcMem]
        mov     rdx,[hbmOld]
        call    [SelectObject]
        mov     rcx,[hbmMem]
        call    [DeleteObject]
        mov     rcx,[hdcMem]
        call    [DeleteDC]

        add     rsp,8 * 9
        ret

 TimerProc1:
        sub     rsp,8 * 4

        mov     [PFT],0                                         ; Zero previous pagefile values
        mov     [PFU],0                                         ;
        mov     [Epfi.cb],sizeof.ENUM_PAGE_FILE_INFORMATION
        lea     rcx,[EnumPageFilesProc]
        lea     rdx,[pContext]
        call    [EnumPageFiles]

        mov     rcx,Gms
        call    [GlobalMemoryStatusEx]

        mov     r8,[Gms.ullTotalPhys]
        lea     rcx,[_V1]
        shr     r8,20
        call    h2a
        mov     r8,[Gms.ullTotalPhys]
        sub     r8,[Gms.ullAvailPhys]
        lea     rcx,[_V2]
        shr     r8,20
        call    h2a
        mov     r8,[Gms.ullAvailPhys]
        lea     rcx,[_V3]
        shr     r8,20
        call    h2a

        mov     r8,[Gms.ullTotalPageFile]               ; Virtual aka Commit value
        lea     rcx,[_V4]
        shr     r8,20
        call    h2a
        mov     r8,[Gms.ullTotalPageFile]               ; Virtual aka Commit value
        sub     r8,[Gms.ullAvailPageFile]               ; Virtual aka Commit value
        lea     rcx,[_V5]
        shr     r8,20
        call    h2a
        mov     r8,[Gms.ullAvailPageFile]               ; Virtual aka Commit value
        lea     rcx,[_V6]
        shr     r8,20
        call    h2a

        mov     r8,[PFT]
        lea     rcx,[_V7]
        shr     r8,8
        call    h2a
        mov     r8,[PFU]
        lea     rcx,[_V8]
        shr     r8,8
        call    h2a
        mov     r8,[PFT]
        sub     r8,[PFU]
        lea     rcx,[_V9]
        shr     r8,8
        call    h2a

        mov     rcx,[mainhwnd]
        lea     rdx,[Client]
        mov     r8d,1
        call    [InvalidateRect]
        add     rsp,8 * 4
        ret

 EnumPageFilesProc:
        mov     rax,[rdx+8]
        add     [PFT],rax                       ; add multiple pagefiles values if they exist
        mov     rax,[rdx+0x10]
        add     [PFU],rax                       ; add multiple pagefiles values if they exist
        mov     eax,1                           ; continue next enumeration
        ret

 h2a:
        sub     rsp,8*4
        lea     rdx,[wsh2a]
        call    [wsprintf]
        add     rsp,8*4
        ret

endf

;----------------------------------------
section '.data' data readable writeable
 mainhwnd       dq ?                      ;;Window Handle
 hDC            dq ?                      ;;Device context handle
 lfhandle       dq ?                      ;;Font Handle
 hbmMem         dq ?                      ;;Memory Bitmap handle
 hdcMem         dq ?                      ;;Memory device context handle
 hbmOld         dq ?                      ;;Old bitmap handle
 Height         dq ?
 Width          dq ?
 pContext       dq ?
 PFT            dq ?
 PFU            dq ?
 hwnd           dq ?

 msg            MSG                       ;;Message structure
 wc             WNDCLASSEX sizeof.WNDCLASSEX,0,WindowProc,0,0,NULL,NULL,NULL,COLOR_BTNFACE+1,NULL,_class,NULL
 PaintS         PAINTSTRUCT
 lfont          LOGFONT

 Client         rt      10,  10, 240,  80

 T1             rt      50,   5, 110,  20
 T2             rt     115,   5, 175,  20
 T3             rt     180,   5, 240,  20
 T4             rt      5,   20,  45,  35
 T5             rt      5,   35,  45,  50
 T6             rt      5,   50,  45,  65

 V1             rt     50,   20, 110,  35
 V2             rt     115,  20, 175,  35
 V3             rt     180,  20, 240,  35

 V4             rt     50,   35, 110,  50
 V5             rt     115,  35, 175,  50
 V6             rt     180,  35, 240,  50

 V7             rt     50,   50, 110,  65
 V8             rt     115,  50, 175,  65
 V9             rt     180,  50, 240,  65

 Gms            MEMORYSTATUSEX
 Epfi           ENUM_PAGE_FILE_INFORMATION

 _V1            rb 20h
 _V2            rb 20h
 _V3            rb 20h
 _V4            rb 20h
 _V5            rb 20h
 _V6            rb 20h
 _V7            rb 20h
 _V8            rb 20h
 _V9            rb 20h


section '.idata' import data readable writeable

 library kernel32,'KERNEL32.DLL',\
                kern,'KERNEL32.DLL',\
                user32,'USER32.DLL',\
                gdi32,'GDI32.DLL',\
                advapi32,'ADVAPI32.DLL',\
                psapi,'PSAPI.DLL'

 include        'api/kernel32.inc'
 include        'api/user32.inc'
 include        'api/gdi32.inc'
 include        'api/advapi32.inc'

 import psapi,\
                GetPerformanceInfo,'GetPerformanceInfo',\
                EnumPageFiles,'EnumPageFilesA'
 import kern,\
                GlobalMemoryStatusEx,'GlobalMemoryStatusEx'    


@AsmGuru62 following heap code on W10 increases pagefile from 3.5GiB to 16GiB before failing. Once closed returns to 3.5GiB
Code:
; Slightly modified 'Revolution' code
;====================================
ALLOC_SIZE = 64 shl 20  ; 64 MiB chunks

include 'win64ax.inc'
.code
begin:
        invoke  GetProcessHeap
        mov     r13,rax
        xor     ebx,ebx
    .again:
        invoke  HeapAlloc,r13,0,ALLOC_SIZE
        cmp     eax,0
        jz      .error
        add     rbx,ALLOC_SIZE
        jmp     .again
    .error:
        sub     rsp,100h
        mov     rsi,rsp
        shr     rbx,20
        cinvoke wsprintf,rsi,'Allocated %u MiB',rbx
        invoke  MessageBox,0,rsi,'HeapAlloc testing',0
        invoke  ExitProcess,0
.end begin    


Description: Running without pagefile and using 4GB VRAM see's loss of use of 5GB of RAM!
Filesize: 2.89 KB
Viewed: 3348 Time(s)

5G.png


Post 22 Jun 2022, 04:45
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.