flat assembler
Message board for the users of flat assembler.

Index > DOS > Allocating large memory in flat mode

Author
Thread Post new topic Reply to topic
Homeground



Joined: 29 Apr 2012
Posts: 12
Homeground 29 Apr 2012, 18:09
Hi,

First post. I haven't been using the flat assembler, I've been using Ketman to learn assembly. But my question is about flat mode. I have a project that needs a large database of maybe 10Mb, but Ketman is a 16-bit app. I've managed to implement Hermann Dullink's "Flat" code, and can read/write in the first Mb using 32-bit pointers. So it all seems to work. But my knowledge of DOS is a bit sketchy, and limited to the basic OS. I know very little about memory managers and how to use them. Given that I can't use anything that puts the system into virtual mode (or else flat won't work), my question is, how do I get my 10Mb?
Post 29 Apr 2012, 18:09
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 29 Apr 2012, 18:19
Well, the whole memory should be there in the address space. Just use it as you like.
Post 29 Apr 2012, 18:19
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
Homeground



Joined: 29 Apr 2012
Posts: 12
Homeground 29 Apr 2012, 18:46
Really? You mean, you don't have to request memory allocation from a memory manager? You can just use all that space without permission?
Post 29 Apr 2012, 18:46
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 29 Apr 2012, 19:27
AFAIK, the flat mode is just that - 32bit mode with all memory accessible on its physical addresses.
Post 29 Apr 2012, 19:27
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 29 Apr 2012, 20:53
The flat real mode just gives you access to memory, but whether this memory is already used by some other program is a different problem. Usually you have some XMS manager in system and you can allocate memory through it (and it guarantees there will be no conflict with any other program as long as it is respecting the XMS), but there are also some other, older methods of signalizing what extended memory is already in use, like the VDISK-style memory location (see MODES.INC in fasm's sources for an example, this method was borrowed from the open-source PMODE extender).
Post 29 Apr 2012, 20:53
View user's profile Send private message Visit poster's website Reply with quote
Homeground



Joined: 29 Apr 2012
Posts: 12
Homeground 29 Apr 2012, 21:51
What other programs could be using the space? If you booted into real mode DOS, your program is the only one running. Unless XMS is doing stuff behind the scenes...

So I'm back to my original question. How do I get my 10Mb and know that I'm not sharing it with some other ghostly user?
Post 29 Apr 2012, 21:51
View user's profile Send private message Reply with quote
freecrac



Joined: 19 Oct 2011
Posts: 117
Location: Germany Hamburg
freecrac 30 Apr 2012, 06:52
Maybe a XMS-Ramdisk-driver is in use?

...

I think it is also recommend to check if our ram is free, or already in use and reserved for the bios.

http://wiki.osdev.org/Detecting_Memory_(x86)

PS: I can only read my answer inside the edit window, but not in the preview window and also not the submitted text. I do not know why.

Dirk

revolution says: Fixed it, now we can see it.
Post 30 Apr 2012, 06:52
View user's profile Send private message Send e-mail Reply with quote
rugxulo



Joined: 09 Aug 2005
Posts: 2341
Location: Usono (aka, USA)
rugxulo 30 Apr 2012, 22:51
Use FASM + WDOSX (+ its WDMEM) and everything should be okay. Of course it can always be more complicated if you let it.
Post 30 Apr 2012, 22:51
View user's profile Send private message Visit poster's website Reply with quote
Homeground



Joined: 29 Apr 2012
Posts: 12
Homeground 03 May 2012, 15:28
I'm looking for something a little more practical. Could someone at least tell me this: there is INT 0x15, AX = 0xE801, which returns contiguous memory blocks above 1Mb. Can I trust that this would be free memory from that point on, that is, until I exit the program, or would I have to keep checking that some other device hasn't decided to start using it?

A second point. I'm still in the process of deciding whether Flat mode is worth using at all. If it is, it might be worth getting to grips with FASM. But at the moment, I don't know, it seems a bit limited. Not only is it confined to DOS, it's confined to DOS minus memory managers. It only works on a very barebones OS.

I've also noticed an anomaly. Trying to read/write with 32-bit regs, I found lods and stos and movs work fine. But not mov reg, [reg]. That instruction seems to load from some other area. At the moment I'm getting instruction codes by planting prefixes 66 and/or 67 in front of normal x86 16-bit codes, assuming that that would make them all work as 32-bit instructions. But it looks like some don't work that way. Is that the case, or have I made an error somewhere?
Post 03 May 2012, 15:28
View user's profile Send private message Reply with quote
Dex4u



Joined: 08 Feb 2005
Posts: 1601
Location: web
Dex4u 03 May 2012, 15:51
If you running under real Dos "INT 0x15, AX = 0xE801", will not work.

It would be helpful if you let us know what your program is for, as there maybe a suitable hobby OS to suit your needs.
Example in DexOS your program is loaded to 8MB you call top of memory function and your program is free to use all memory from 8mb to top of memory (all 32bit usable ram, minus any drivers loaded) returned by calling the function.

Theres also dos2x that does the same as DexOS, but runs on top of Dos and lets you use most Dos ints etc too, from PM.
Post 03 May 2012, 15:51
View user's profile Send private message Reply with quote
Homeground



Joined: 29 Apr 2012
Posts: 12
Homeground 03 May 2012, 16:38
My program is an editor of sorts, which needs a large database containing as many Elizabethan documents as I eventually get round to downloading. It would include all the plays of Shakespeare, Marlowe, Jonson, and several others, as well as poems and pamphlets from the period. The version I have at the moment works purely in 16-bit real mode, and the database consists of files of size 8000h bytes stored on disk and loaded sequentially, if need be, using normal DOS load/save services. Obviously using disk-swap techniques instead of memory slows things down, and it would be great to have the whole database loaded at +1Mb. I could have gone the PM route, but that would mean abandoning Ketman (an app I know and like), so I thought flat mode working off Ketman would be a good compromise. And I do want to do it in assembly language, by the way, not in C or Pascal or anything else.

But it's turning out a bit more complicated than I thought. I've been experimenting since my last post, and I've discovered that putting 66/67 codes in front of mov reg, [bp] makes it read from [si] and mov reg,[bx] reads from [di]. I haven't found out all the others yet, but I'm in it now for the long haul if only I can secure memory and have it safe. Is there no way of doing that in flat mode? If int 15/ax=E801 doesn't work, what does?
Post 03 May 2012, 16:38
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 03 May 2012, 16:45
Homeground: You shouldn't be putting manual 0x66/0x67 prefixes in your code. Let the assembler do its job and everything will work fine:
Code:
use16
mov eax,[esi] ;assembler automatically puts prefixes when needed
;...
use32
mov eax,[esi] ;assembler doesn't put prefixes when they are not required    
Post 03 May 2012, 16:45
View user's profile Send private message Visit poster's website Reply with quote
Homeground



Joined: 29 Apr 2012
Posts: 12
Homeground 03 May 2012, 16:54
revolution wrote:
Homeground: You shouldn't be putting manual 0x66/0x67 prefixes in your code. Let the assembler do its job and everything will work fine:
Code:
use16
mov eax,[esi] ;assembler automatically puts prefixes when needed    


As I said, Ketman is a 16-bit app. It doesn't even recognize mov eax,[esi].
Quote:

use32
mov eax,[esi] ;assembler doesn't put prefixes when they are not required

Surely, the only reason they wouldn't be required is that you are running in PM?


Last edited by Homeground on 03 May 2012, 16:56; edited 1 time in total
Post 03 May 2012, 16:54
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 03 May 2012, 16:54
Homeground wrote:
So I'm back to my original question. How do I get my 10Mb and know that I'm not sharing it with some other ghostly user?
With DOS the user is expected to know everything running in the system. DOS is not a protected OS which means it won't help you to stop clashes. So basically, only you can determine what other code might be running by examining the autoexec.bat and config.sys files (and whatever other startup files DOS might use).
Post 03 May 2012, 16:54
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 03 May 2012, 16:57
Homeground wrote:
Surely, the only reason they [prefixes] wouldn't be required is that you are running in PM?
Just tell fasm what mode the code is running in with either use16 or use32 and everything will be assembled correctly. You don't need to put in manual prefixes.
Post 03 May 2012, 16:57
View user's profile Send private message Visit poster's website Reply with quote
freecrac



Joined: 19 Oct 2011
Posts: 117
Location: Germany Hamburg
freecrac 04 May 2012, 13:28
http://files.osdev.org/mirrors/geezer/osd/ram/index.htm#layout
Quote:
Determining extended memory size is surprisingly difficult:

* Use BIOS calls:
1. INT 15h AX=E820h (32-bit CPU only). If this fails...
2. ...use INT 15h AX=E801h. If this fails...
3. ...use INT 15h AH=88h.

http://wiki.osdev.org/Memory_Map_(x86)

(@revolution: Thanks for fixing my postings. If i do something wrong, please let me know.)

Dirk

revolution says: fixed url encoding
Post 04 May 2012, 13:28
View user's profile Send private message Send e-mail Reply with quote
Homeground



Joined: 29 Apr 2012
Posts: 12
Homeground 04 May 2012, 21:29
Those are good links. Thanks.
Post 04 May 2012, 21:29
View user's profile Send private message Reply with quote
Dex4u



Joined: 08 Feb 2005
Posts: 1601
Location: web
Dex4u 05 May 2012, 11:19
The problem is when Dos boots, it hooks into these ints and reports less than a MB.
Under win98 or xp this maybe be different ?.
But from my test under real Dos (freeDos), it does not give true ram available.
Post 05 May 2012, 11:19
View user's profile Send private message Reply with quote
rugxulo



Joined: 09 Aug 2005
Posts: 2341
Location: Usono (aka, USA)
rugxulo 05 May 2012, 13:14
Homeground wrote:
I'm looking for something a little more practical.


Eh? Seriously, you can do whatever you want, esp. if you're just learning. But if you use an extender like WDOSX, you don't have to worry about segment overrides or detecting memory or playing nice with memory managers or any of that goop. You just write, and it just works in almost all situations.

I tried to write up a simple example, but perhaps it's not too useful. Nevertheless, it should prove the point, as it does work. (BTW, you don't have to use WFSE, but for laughs I did, esp. to prove that you could bind the data file to the extender. But that is for PE format only, so if you want something else [why? WDOSX makes it DOS friendly], it won't support that. And honestly, ~3 MB .EXE is fairly big, heheh, [though compressed, originally ~6 MB!!] but simplifies deployment of only one file, nifty right?)

P.S. EXE is ~3 kb with no extender (but won't run!), ~12 kb with extender, everything else is data. So you could presumably keep the data in separate file(s) if you really wanted a slim binary. Use int 31h, 501h (DPMI) to dynamically alloc extra memory (use 500h to detect how much available) if not statically allocating at assembly time, and use WDMEM.EXE if you need to shrink resident .EXE size if shelling out to call another app.

http://www.gutenberg.org/files/1581/1581.zip (1.8 MB .ZIP, aka, douayrheims.txt)

Code:
; fasm bible.asm (FASM 1.70.01)
; stubit bible.exe (WDOSX 0.97)
; wadd bible.exe douayr~1.txt douayrheims.txt (from Project Gutenberg)

format pe

section '.text' code readable executable

Komenco:
  mov ah,9
  mov edx,bar
  int 21h
Check_WFSE:
  mov eax,0FFFDh
  stc
  int 21h
  jc Fino
  cmp eax,'ESFW' ; WFSE
  jnz Fino
Open:
  mov eax,3D00FFFDh
  mov edx,bible
  int 21h
  jc Fino
  mov [handle],ax
SeekStart:
  mov eax,4200FFFDh
  mov bx,[handle]
  mov ecx,4000000 ; skip to middle of big file
  movzx edx,cx
  shr ecx,16
  int 21h
  jc Close
Read:
  mov eax,3F00FFFDh
  mov bx,[handle]
  mov ecx,70000 ; bigger than 64 kb
  mov edx,textdata
  int 21h
  jc Close
DisplayText:
  mov ebx,67930 ; find our Amos quote
  mov [textdata+ebx+55+1],'$'
  mov ah,9
  lea edx,[textdata+ebx] ; say quote
  int 21h
Close:
  mov eax,3E00FFFDh
  mov bx,[handle]
Fino:
  mov eax,4C00h
  int 21h

section '.data' data readable writeable
  bar db '=======Amos 3:4=="Will a lion..."=='
      db '===================================',13,10,'$'
  bible db 'douayrheims.txt'
  handle dw 0

section '.bss' data ; automatically allocs this much at runtime
  textdata rb 70000

section '.reloc' fixups data readable discardable ; WDOSX needs relocs
    
Post 05 May 2012, 13:14
View user's profile Send private message Visit poster's website Reply with quote
nop



Joined: 01 Sep 2008
Posts: 165
Location: right here left there
nop 11 Jun 2012, 01:49
Homeground wrote:
Really?.... You can just use all that space without permission?
whose cpu is it anyway just do with it what you like no permision needed Very Happy
Post 11 Jun 2012, 01:49
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.