flat assembler
Message board for the users of flat assembler.

Index > OS Construction > EXECUTION ENVIRONMENT

Author
Thread Post new topic Reply to topic
edfed



Joined: 20 Feb 2006
Posts: 4240
Location: 2018
edfed
Sorry, but i need the most popular idea to manage the functions in a x86 computer environment.
For that i need your opinion. Opinion of asm coders.

passing parameters in fixed memory location?
(simplest but not good for multiple calls in multitask environment)
passing parmeters in registers?
passing parameters by a stack?
passing parameters by tables indexed by registers?
passing parameters by tables indexed by indexes in stack?
passing parameters by tables indexed by registers and segments registers?
etc...

By reading the manuals i see that all theses shemes are possible.
But the problem is to find the best and simplest to use in fasm ide
Why in fasm ide? Cause is the best for coding and have the total control of the machine.

How to access the functions?
If they are in the source code of the application, it is simple but, what about api that are out of the code?
Invoke seems to be a good way, but it is not really asm

I want to call directlly the function
But how to make this with an extern function?
In an other segment, at a various offset
because each machine don't load the functions at the same memory location
So it needs a system function called during the launch process of an application
But how can i say to the system the functions needed?
by a name in an header?
It complicates the development process
Post 08 Nov 2007, 23:23
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid
Quote:
passing parameters in fixed memory location?
(simplest but not good for multiple calls in multitask environment)

+ most simple
- impossible to multithread or shared code - completely bad for OS API functions
- mess to keep variables declared in separate place
- waste of memory - much more memory used than needed
- can't use recursion, eg. can't be the ONLY calling standard
- you must come up woth LOTS of names Smile
- need to access externa variables when calling external functions
- slower than stack

Quote:
passing parmeters in registers?

+ can be fast sometimes (most of times it isn't)
- limited number of parameters
- harder for asm coder to call it (must preserve registers)
- bad for "higher level" functions, which just call series of other function and process their return values. Such functions make majority of most bigger applications.
- harder to call functions - you must remember which register holds what

Quote:
passing parameters by a stack?

+ fastest
- not easy to access without macros
- stack frame needed (or if not, even harder to access)

others are just some variations of these.

Quote:
Why in fasm ide? Cause is the best for coding and have the total control of the machine.

What does development enviroment has to do with control of machine?

Quote:
If they are in the source code of the application, it is simple

using single sources has downsides too:
- unscessary slow compilation of big apps
- worser reusability of code

Quote:
what about api that are out of the code?
Invoke seems to be a good way, but it is not really asm

you mean you don't like indirect call? (eg "call [func]" instead of "call func")

Quote:
I want to call directlly the function
But how to make this with an extern function?

same way like everyone does, with an "extrn" for example?
Code:
format MS COFF
extrn func
call func    

as for API functions, you can relocate code upon loading, if you want direct calls so much.

Quote:
In an other segment, at a various offset

why do you want segmentation in your OS. What for? How about x86-64 machines that don't really support segmentation?
Post 09 Nov 2007, 00:47
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4240
Location: 2018
edfed
Segmentation works like i hope!
It brings separates memory zones

for graphic, we need a segment for diect screen memory, an other for screen buffer, and others for microbuffers (instance buffer)

for audio is the same

for 3d is the same

for file system is the same

and to import functions that are not system, like dll, we need a multisegment multitread os

for x86-64 machines, the problem is not the same and i don't care for the moment, my best machine is a PIII @ 800MHz !!!

"extrn func" ???
seems to be good

but how does it works exactlly?
EXTRN ???


To speak again about segmentation, it's true that i don't need more than one segment for the os, 4G limit, etc...
but for more evolutivity i want to support multi segment os. Even it is a bit slower.

I want to employ all the features of a x86 processor from 386 to P4
64 bit seems to be too much
The half of the memory is wasted!

the fasmw ide have only one limit, the coders knowledge!
Post 09 Nov 2007, 11:23
View user's profile Send private message Visit poster's website Reply with quote
tom tobias



Joined: 09 Sep 2003
Posts: 1320
Location: usa
tom tobias
Two first impressions:
1. Thanks (again) edfed, for another very important and interesting question on the FASM forum....
2. Thanks (again) vid, for a sparkling and genuinely informative reply, I really enjoyed reading it. Your explanation taught me many things, which I had either forgotten, or never understood in the first place. Well done.
These questions which follow, then, are written in the spirit of further clarifying some of the mysteries which accompany this important topic: how best to pass data from one routine (= thread, = process, = function) to another? I wish to focus on only one of vid's excellent explanations:
Analyzing the features associated with passing parameters by a fixed memory location, vid wrote:

+ most simple
- impossible to multithread or shared code - completely bad for OS API functions
- mess to keep variables declared in separate place
- waste of memory - much more memory used than needed
- can't use recursion, eg. can't be the ONLY calling standard
- you must come up woth LOTS of names
- need to access externa variables when calling external functions
- slower than stack

Two of these points (slow speed, compared with "stack", and ostensible waste of memory) I consider, in my own, (some would argue) distorted, world view, to be completely false, or at least, inaccurate, hence, irrelevant. One point (requires "LOTS of names") seems to me to have missed the point entirely, since the memory location need not represent the size of a single operand, but could be a gigabyte in size, with no names at all. I disagree fundamentally with the notion that use of memory to pass parameters is associated with a bureaucratic nightmare: "mess to keep variables declared in separate place". The problem is not fundamentally different from the task of the postal service keeping track of our mail--a chore, a task, but not insurmountable. Regarding the contention that it is "impossible to multithread", if one passes parameters by memory location, I think Dijkstra, among many others, http://www.thocp.net/biographies/dijkstra_edsger.htm
has shown this to be not correct. Perhaps I am in error on this aspect of computer science history, if so, please correct me. There remain then, two points, about which I remain unconvinced.
1. "need to access external variables when calling external functions." I am not arguing against this notion, but I simply fail to comprehend why this particular aspect of using memory to pass parameters is distinctive or unique, when compared with using cache (aka "stack") or a register. In other words, in my ignorance, it would seem to me, that the chore of accessing "external" variables, is not limited to that single circumstance when one employs memory to pass parameters, but is also extant as an obstacle, when using either of the other two methods: stack or register.
2. "can't use recursion..." This is a very important point, in my opinion. I am an enthusiastic disciple of using memory to pass parameters, and an ardent opponent of using either "stack" or registers for this purpose. I would hope that vid's contention that recursion is incompatible with passing parameters by memory, could be amplified a bit. I confess that I dislike both indirection, and recursion, nevertheless, I do understand that both are much beloved by the orthodox programming community, and so, I would like to learn WHY recursion (a function calling itself, repeatedly, until some internal activity is complete) is incompatible with using memory to pass parameters. I suspect that most programmers envision use of the stack to accomplish recursion, and I would then repeat the obvious, a stack is nothing more than memory. Accessing a stack frame is not nearly as convenient, in my opinion, as fetching data from a calculated offset within memory. It is not simply a question of bookkeeping, it is also a question of readability. For me, readability trumps everything else, especially, speed of execution, (stack based instructions are supposedly faster in execution), and the widely proclaimed reduction in memory useage. We have gigabytes of memory just sitting there, doing nothing, so I am unconcerned with saving it. Execution speed: 99% of the code running out there is finished in microseconds, with the user completely unaware that his cpu is just sitting there, twiddling its thumbs, doing nothing. We don't need programs to run FASTER, we need programs that can be modified readily, and stack based programming is at best, arduously deciphered.
Smile
Post 09 Nov 2007, 11:27
View user's profile Send private message Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid
Quote:
One point (requires "LOTS of names") seems to me to have missed the point entirely, since the memory location need not represent the size of a single operand, but could be a gigabyte in size, with no names at all. I disagree fundamentally with the notion that use of memory to pass parameters is associated with a bureaucratic nightmare: "mess to keep variables declared in separate place". The problem is not fundamentally different from the task of the postal service keeping track of our mail--a chore, a task, but not insurmountable.

the caller must know *where* in this chunk of memory to write the value, and called function must know where in that chunk to look for it. Keep in mind that if such system will be developed through time, these places are very likely to change with different versions of system. How to deal with that, except using symbolic names to refer these places?

Quote:
Regarding the contention that it is "impossible to multithread", if one passes parameters by memory location, I think Dijkstra, among many others, http://www.thocp.net/biographies/dijkstra_edsger.htm
has shown this to be not correct. Perhaps I am in error on this aspect of computer science history, if so, please correct me. There remain then, two points, about which I remain unconvinced.

You are in error. If function takes parameter in fixed memory location, it cannot be used safely in multithreaded application.

Quote:
1. "need to access external variables when calling external functions." I am not arguing against this notion, but I simply fail to comprehend why this particular aspect of using memory to pass parameters is distinctive or unique, when compared with using cache (aka "stack") or a register. In other words, in my ignorance, it would seem to me, that the chore of accessing "external" variables, is not limited to that single circumstance when one employs memory to pass parameters, but is also extant as an obstacle, when using either of the other two methods: stack or register.

i think you didn't get the point here. If there is API function ExitProcess, which takes argument in fixed-place variable "exit_code", then you need to import two external symbols, instead of one in case of stack passing. Also accessing to variable either has to be done this way:
Code:
mov eax, [exit_code_address]
mov [eax], -1
call [ExitProcess]
    
or you have to relocate code.

regarding recursion, very simple example of readability:
Code:
proc fibonacci value
push ebx

mov eax, [value]
cmp eax, 1
jbe .r

dec eax
push eax
call fibonacci
mov ebx, eax

mov eax, [value]
sub eax, 2
call fibonacci

add eax, ebx

.r:
pop ebx
ret
endp
    

can you provide me with more readable version using static variables? Only way you co do this is to make your own implementation of stack-like structure, which has many disadvantages against machine-implemented one (slower, much harder (and less readable) to operate, need to manually check overflow after every operation, harder access to top of stack, ...)
Post 09 Nov 2007, 12:12
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid
segmentation for OS is a very bad idea, if you want to ever port some application for that OS. You don't NEED to use it. You can decide to

Quote:
"extrn func" ???
seems to be good

but how does it works exactlly?
EXTRN ???

Study something about compiling to objects and using linker. Too much to write about here.


Quote:
To speak again about segmentation, it's true that i don't need more than one segment for the os, 4G limit, etc...

it's not true, you need more of them (code, data, task)

Quote:
but for more evolutivity i want to support multi segment os. Even it is a bit slower.

evolutivity?

Quote:
I want to employ all the features of a x86 processor from 386 to P4

do you have some idea about what features are there?
Multithread support, multiple cores, debug registers, last branch, VMX, V86, SMM, 32bit protected mode outside IA-32e mode, 32bit protected mode in IA-32e mode, and don't forget that 64bit mode which you don't want to support is feature of x86 processor too.

Quote:
64 bit seems to be too much
The half of the memory is wasted!

have you got any idea how x86-64 works, or are you just quessing?
Writing about things you have no idea about probably isn't gonna make you more respected or popular here...

Quote:
the fasmw ide have only one limit, the coders knowledge!
again, what the hell has the IDE to do with it? What changes if I use notepad, VIM, hexeditor, or whatever to edit sources?
Post 09 Nov 2007, 12:27
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4240
Location: 2018
edfed
i know how works all the processors i work on
and more...

my goal, if you want to know, i to integrate Pentiums on an electronic hand-made circuitry different of a computer

i know all the new features of all different x86 processors, and for all things i don't know, i can read the manuals.

x86_64 is a big waste of memory!
64 bit operands for addressing!! 4G*4G

64bit maths is ok, but i don't see why addressing in 64bits even it is interesting to address more than 4G

why they don't create a realmode32?
working like realmode but whith extension of addressing in segment registers like in realmode?
supporting realmode code?

the technoloogy don't permitts to have 2^64 bytes of RAM onboard
and the waste of memory is significant on boards that have only 2^32 bytes of memory or less

the virtual memory is on drive, but no drive can contain 2^64bytes

i will write an os accordinglly to all that you say on this forum.
i read the algorythms you write, the little parts of code
and after i rewrite in my style. optimising, modifying, like a technique, not like a copyrighted code.

for exemple, i always employ the memory location to enclose the parameters,

Code:
;edi is the parent item
;esi is the current item
caller:
        mov eax,[esi+item.size]
        xor ebx,ebx
.next:
        push ebx eax esi
        mov edi,esi
        mov esi,[edi+ebx*4+item.data]
        cmp esi,0
        je @f
        mov eax,[esi+item.call]
        cmp eax,0
        je @f
        call eax
@@:
        pop esi eax ebx
        inc ebx
        sub eax,4
        jg .next
.end:
        ret 
    


the problem is to translate this in PM!
Post 09 Nov 2007, 13:23
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4240
Location: 2018
edfed
Code:
desktop:
dd caller,0,0,320,200,@f-$-4
dd printcurs
dd about
dd textview
dd taskbar
dd startbar
dd optionmenu1
dd optionmenu2
dd optionmenu3
dd optionmenu4
dd exitor
dd mouse.pointer
@@:   
    

this is the main object, the desktop

Code:
dd caller,0,0,320,200,@f-$-4    

this is the common header for all graphic based object
the first dword is the function address of current object
the four following dword is the box that contain the object
0,0,320,200 is the box of mode13h
$-@f-4 is the count of bytes that follow
dd XXX are adress of son object,

for exemple :
Code:
printcurs:
dd caller,260,5,0,0,@f-$-4
dd CURSX
dd CURSY
@@:
CURSX:
dd printvalue,0,0,0,0,@f-$-4,curs.x,0
db blue+20,yellow+20,0,0
@@:
CURSY:
dd printvalue,0,10,0,0,@f-$-4,curs.y,0
db blue+20,yellow+20,0,0
@@:           
    

this works very well but only in real mode!!
and without segmentation, to contain segment i need 2 bytes more per segments
Post 09 Nov 2007, 13:37
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4240
Location: 2018
edfed
just imagine this in 64 bits!!
Post 09 Nov 2007, 13:39
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid
Quote:
i know how works all the processors i work on
and more...

i know all the new features of all different x86 processors, and for all things i don't know, i can read the manuals.

so, DO you know all the new features, or you don't? Even someone who haven't seen a computer in his life can state "i know all features of x86 processors, and those which i don't know i can learn from manual", and it would be just as true. Do you know how to work with VMX? Do you know how to use SMM? Do you know how to establish v86 mode? Do you know how to set up and synchronize multiple cores? Do you know how to control hyperthreading? Can you use MTRRs? These are new features of x86.

Quote:
my goal, if you want to know, i to integrate Pentiums on an electronic hand-made circuitry different of a computer

just processor, or entire motherboard?

Quote:
x86_64 is a big waste of memory!
64 bit operands for addressing!! 4G*4G

only part where 64bit addressing wastes memory is when you store pointers. Not 100% of application memory holds stored pointers, so yours "double memory" claim wasn't true. maybe 20% increase, at most.

Quote:
64bit maths is ok, but i don't see why addressing in 64bits even it is interesting to address more than 4G

as you surely know, most of addressing is done using RIP-relative addressing, which has 32bit displacement.

Quote:
why they don't create a realmode32?
working like realmode but whith extension of addressing in segment registers like in realmode?
supporting realmode code?

because it will be vastly inferior to current design. realmode doesn't have any memory protection, any kernel protection from user mode part, etc. If you would connect your real-mode computer to internet, it wouldn't last a minute before being malwared.

Quote:
the technoloogy don't permitts to have 2^64 bytes of RAM onboard
and the waste of memory is significant on boards that have only 2^32 bytes of memory or less

Not true, there are systems that require, and have, more than 4GB, and where RAM matters.
Also, keep in mind that 64bit address space is still useful, even if you have less than 4GB RAM. For example it allows mapping big files into memory.

Quote:
i will write an os accordinglly to all that you say on this forum.
i read the algorythms you write, the little parts of code
and after i rewrite in my style. optimising, modifying, like a technique, not like a copyrighted code.

depends on definition of "operating system", but do you realize that writing anything that can really be called operating system nowadays will take you at least 5 years?

do you plan your OS to support multitasking?

Quote:
this works very well but only in real mode!!

maybe i don't understand your design, but ... for what reason exactly it doesn't work in protected mode?
Post 09 Nov 2007, 13:54
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4240
Location: 2018
edfed
5 years, it's ok 2012, 30 years old!
i plan to support multitask, v86, HT, multicore, etc etc...
now i have to integrate all the real mode functions i've made into PM, but it's not simple cause of the system memory manager, dynamic memory allocation is a hard trick, the paging is really a good thing but use a lot of memory...

5 years???
i say 2 mounthsqor the 0.0001 version
a simple .bin file to write on a boot disk!!!
Post 09 Nov 2007, 14:11
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid
Quote:
5 years, it's ok 2012, 30 years old!
i plan to support multitask, v86, HT, multicore, etc etc...

10 years then, at least Smile

Quote:
i plan to support multitask

then forget about passing parameters in static memory variables. think about case when two tasks are going to call same API at "same time".

by the way, you didn't answer me which of those new features you know, and which you dont Wink

Quote:
now i have to integrate all the real mode functions i've made into PM, but it's not simple cause of the system memory manager, dynamic memory allocation is a hard trick

yeah, it is very hard if it has to be done right.

the paging is really a good thing but use a lot of memory...[/quote]
i hope you plan to support paging too, it is one of those more useful features of x86, taken for granted in every modern OS.

do you also plan to implement memory protection? eg. normal application has no chance to hook kernel, and take over computer?
Post 09 Nov 2007, 14:47
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4240
Location: 2018
edfed
all features of pentium III @ 800MHz will be supported, multiprocessor system too, but later.

For the moment i just plan on the operating system common process and structures. i seek the simplest for me.
using the page table and gdt for memory mapping

1 application = 1 ldt
IDT will be user modifiable
TSS will be unemployed in hardware task switch
this hardware switch will be undone
just a periodic software task switch will be done
the system will load applications and seek the needed imported functions stored in executable file during compile
each application will possess a ldt witch holds the entire needed memory for application and a page table that is dynamic

the task of the os is to allocate the memory in the ldt
and the paging will be used during execution
if a file is growing in size , a page can be added
and the segment is expanded

it's a spaggheti plate, :-S
Post 09 Nov 2007, 15:20
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid
Quote:
IDT will be user modifiable

so for example application can completely stop system timer, and thus grant 100% CPU for itself? that isn't good multitasking.

Quote:
the system will load applications and seek the needed imported functions stored in executable file during compile

you mean, compiler will construct list of dependencies for every executable?

Quote:
each application will possess a ldt witch holds the entire needed memory for application and a page table that is dynamic

including system API procedures?
Post 09 Nov 2007, 15:32
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4240
Location: 2018
edfed
ouch!
Post 09 Nov 2007, 15:51
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4240
Location: 2018
edfed
i think i will primarily make a multitask environment like if it was a multiple chip environment

one function is one chip

an application then is a board
and a good engineer knows the chips he use
i will give two manners for function integration
simply include the function and then call it (it's in the code segment of the application and is a code/data segment?)
declare a list of used library that are included
inclusion of a library will consist of a equate file and an address list file that correspond to the location of the desired functions loaded in the ring3 function code segment of the os, ---> far call
(a little idea is growing, data stack for dynamic access)

how to implement this....
let's go

paging required at start up now
direct to disk access required
page to disk transfert required

code in ring3 need to call code on ring 0
or i will simply use a code segment at ring3
and ring1 ring2?
it is the next problem
Post 09 Nov 2007, 16:07
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4240
Location: 2018
edfed
i've made some thing that seems to be like i want
Post 10 Nov 2007, 23:55
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on YouTube, Twitter.

Website powered by rwasa.