flat assembler
Message board for the users of flat assembler.

Index > OS Construction > 64-bit Flat Real Mode is possible!!!

Author
Thread Post new topic Reply to topic
neville



Joined: 13 Jul 2008
Posts: 507
Location: New Zealand
neville 31 Mar 2014, 19:06
I've discovered a way to return to real mode with 64-bit selectors intact after setting them up in long mode. I discovered it by an amazing accident, and it's so obscure I think it's probably an unpublished mode left in the silicon by the chip makers, presumably for in-house testing.

Both 32-bit and 64-bit code segments are possible but so far it appears that a 16-bit code segment is not possible Sad This is unfortunate since it becomes much more difficult to access 16-bit BIOS calls. Unfortunately this could limit the usefulness of this mode. However, I'm still hopeful of discovering a way to set up 16-bit code, maybe via Compatibility Mode within Long Mode. That would certainly make a 64-bit version of FAMOS a lot easier to implement Smile

The other major issue is that it seems to be processor dependent. My very limited testing so far (with 2 different CPU's - one AMD and 1 Intel) shows the AMD CPU ends up with true linear memory like normal 32-bit big real mode, but the Intel CPU still has some paging dependencies, dependent on the page tables set up in long mode. In a weird sort of way the CPU seems to think paging is still enabled in real mode!

I haven't decided what name should be given to this mode, but some possibilities are Super Real Mode (SRM), Long Real Mode (LRM), Long Unreal Mode (LUM), Long Big Real Mode (LBRM) or maybe just "Real64".

I'm not ready to publicly release my source code yet, as there is still much testing to do and (hopefully) compatibility issues to solve. So it would be great to get it tested on as many different CPU's as possible.

So if any fasm board members are interested, please PM me and I will provide a binary for testing purposes. Just let me know exactly what processor(s) you have available for testing when you PM.

_________________
FAMOS - the first memory operating system
Post 31 Mar 2014, 19:06
View user's profile Send private message Visit poster's website Reply with quote
sid123



Joined: 30 Jul 2013
Posts: 339
Location: Asia, Singapore
sid123 01 Apr 2014, 01:13
However remember that Long Mode ignores limit fields of most registers instead of filling the descriptor cache. So whatever limits you'll set LM will simply ignore it. The other problem is that some computers need the CR0.PG bit to be enabled for accessing 64bit extensions (R8-R15, XMM7-XMM15 etc.), switching back to Real Mode (on Intel processors) with CR0.PG enabled will cause an exception (IIRC).
The only thing I've seen close to 64-bit Unreal Mode is disabling an MSR (in AMD CPUs) which determines the 32-bit Address wrap, you can disable the MSR and have 8 GB of Memory Access through 2 segments in Unreal Mode. I'm not sure if Intel has this.
Maybe neville will do something beyond my imagination...... Wink
Post 01 Apr 2014, 01:13
View user's profile Send private message Reply with quote
Feryno



Joined: 23 Mar 2005
Posts: 509
Location: Czech republic, Slovak republic
Feryno 01 Apr 2014, 05:42
Did also FS base and GS base (MSRs C0000100, C0000101) survive the mode switch? Both at AMD64 as well Intel?
Post 01 Apr 2014, 05:42
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
neville



Joined: 13 Jul 2008
Posts: 507
Location: New Zealand
neville 01 Apr 2014, 22:39
Ok so I FOOL'd some people Smile It was just my little April Fool's joke. I won't name those who PM'd me, but you know who you are! Is April Fool's Day recognised all around the world? One person who lives within a few hours of my time zone PM'd me but was obviously very suspicious Very Happy

But all joking aside, I DO wish Real64 was possible! Why shouldn't our PC's have 64-bit CPU's that can access any 64-bit physical address via simple 64-bit direct addressing Question Just like we've always had since the 8088.

_________________
FAMOS - the first memory operating system
Post 01 Apr 2014, 22:39
View user's profile Send private message Visit poster's website Reply with quote
neville



Joined: 13 Jul 2008
Posts: 507
Location: New Zealand
neville 01 Apr 2014, 22:40
sid123 wrote:
Maybe neville will do something beyond my imagination...... Wink
Alas, not yet Smile Feel free to post your links or references about the MSR disabling trick though!

_________________
FAMOS - the first memory operating system
Post 01 Apr 2014, 22:40
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20298
Location: In your JS exploiting you and your system
revolution 01 Apr 2014, 23:44
neville wrote:
But all joking aside, I DO wish Real64 was possible! Why shouldn't our PC's have 64-bit CPU's that can access any 64-bit physical address via simple 64-bit direct addressing Question Just like we've always had since the 8088.
Probably because it is not a mode that is wanted by the "community". But you could always make your own CPU (either an ASIC or from an FPGA) and do whatever you wanted with addressing and ISA.
Post 01 Apr 2014, 23:44
View user's profile Send private message Visit poster's website Reply with quote
sid123



Joined: 30 Jul 2013
Posts: 339
Location: Asia, Singapore
sid123 02 Apr 2014, 10:15
hi Neville,
If you want to simply run BIOS Interrupts in 64bit Mode you could either write an emulator (highly not recommended) or execute the code inside an existing emulator (libx86emu), or better make use of hardware virtualization, as I see it, you could simply track any faults happening due to code running inside the VM and handle them accordingly, it would be cool to see such a thing.
As for the MSR I had been searching the whole day for that, and finally found it in the AMD Kernel and BIOS Developer Guides:
Roll down to page 459 and you'll see this:
AMD Kernel and BIOS Developer Guides wrote:
Wrap32Dis: 32-bit address wrap disable. Read-write. Reset: 0. 1=Disable 32-bit address wrapping. Software can use Wrap32Dis to access physical memory above 4 Gbytes without switching into 64bit mode. To do so, software should write a greater-than 4 Gbyte address to MSRC000_0100 [FS Base (FS_BASE)] and MSRC000_0101 [GS Base (GS_BASE)]. Then it would address ±2 Gbytes from one of those bases using normal memory reference instructions with a FS or GS override prefix. However, the INVLPG, FST, and SSE store instructions generate 32-bit addresses in legacy mode, regardless of the state of Wrap32Dis

Sadly it's specific to AMD CPUs I'm not sure why Intel doesn't have this.
Post 02 Apr 2014, 10:15
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20298
Location: In your JS exploiting you and your system
revolution 02 Apr 2014, 10:42
sid123 wrote:
I'm not sure why Intel doesn't have [Wrap32Dis].
Maybe because having to use segment registers for addressing would cause nightmares for the devs when they think back to the bad old days of 8086 segmentation and all the problems and headaches it caused. I still live in hope that one day the x86 design line will eventually eliminate segments registers completely. Maybe the x86-128 line will get us there?
Post 02 Apr 2014, 10:42
View user's profile Send private message Visit poster's website Reply with quote
neville



Joined: 13 Jul 2008
Posts: 507
Location: New Zealand
neville 02 Apr 2014, 19:31
revolution wrote:
Maybe because having to use segment registers for addressing would cause nightmares for the devs when they think back to the bad old days of 8086 segmentation and all the problems and headaches it caused. I still live in hope that one day the x86 design line will eventually eliminate segments registers completely.
I share your disdain for segment registers - but they were invented by Intel in the first place. And it's not as if Intel haven't copied AMD initiatives before Wink It was an imperfect solution to the "problem" of how to address more than 64K of memory when you only have 16-bit address registers in the silicon. Several more obvious solutions were overlooked, but maybe they foresaw the need for selectors in Protected Mode from the 80286 onwards Very Happy

However, you still seem to be happy with non-flat memory addressing and enforced paging with the associated problems, headaches and nightmares of Page Tables etc.? Virtual Memory addressing is fine, as an optional mode, but why shouldn't non-segmented flat memory also be at least an option (if not the default)?

@sid123 Thanks for your research. I must try it on my AMD64 cpu one day.

_________________
FAMOS - the first memory operating system
Post 02 Apr 2014, 19:31
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20298
Location: In your JS exploiting you and your system
revolution 02 Apr 2014, 23:46
neville wrote:
... but why shouldn't non-segmented flat memory also be at least an option (if not the default)?
You asked the same question. Do you really expect a different answer? It is because the "community" don't need such a thing. It doesn't fit with the multitasking and dynamic memory allocation of mainstream OSes. Chip makers won't dedicate extra silicon to something they see as not being used. Including it requires testing and support resources which costs money. If you really feel you have a case for its inclusion then you need to convince AMD and/or Intel to put in the time and allocate budgets and whatnot. Or just make your own CPU and give the finger to the big chip makers.
Post 02 Apr 2014, 23:46
View user's profile Send private message Visit poster's website Reply with quote
neville



Joined: 13 Jul 2008
Posts: 507
Location: New Zealand
neville 04 Apr 2014, 05:40
revolution wrote:
Do you really expect a different answer?
Not from you, obviously, but your opinion is not the only valid "answer". Unless of course you have some mandate to write on behalf of all fasm board members (except myself)!

Let's Get Real here (pun intended):
1. Microsoft et al pay the piper, so they call the tunes
2. Deliberate architectural complexity provides the means to protect big corporate interests
3. An elegantly-simple flat memory model as per my description would not be on the big corporate's wish list
4. Multitasking and dynamic memory allocation per se have nothing to do with it: both can be implemented in a simple flat memory model
5. Making a one-off CPU would be a completely pointless exercise for very obvious reasons. (So pointless and so obvious, one wonders why it would even rate a mention!)
6. AMD and Intel will only ever be convinced of anything by their fellow big corporates. See 1 above....

_________________
FAMOS - the first memory operating system
Post 04 Apr 2014, 05:40
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20298
Location: In your JS exploiting you and your system
revolution 04 Apr 2014, 07:32
Some good reasons above.

1. Probably true. And they make up the "community". Although I would be less circumspect if the "et al" was fleshed out to more than just picking on MS.
2. I can't agree here. The complexity is mostly historical. Lots of companies have made alternative CPUs with simpler implementations. Easier to manage and test etc. means less overheads.
3. Definitely agree. Large companies don't like to support unsecurable OSes. Note, that doesn't mean the current OSes are secure, just that in principle they could be secured if the code weren't so hard to get right. But flat without paging or protection is not on any major players list.
4. Also true. But without process isolation things are less reliable and the public get upset when program A kills all other processes and the OS because the coder of A was careless.
5. You don't have to make a one-off. If your CPU design is popular other people can use it. Many companies have started in this way.
6. True. And that is why they won't invest time, effort and money if they feel there is not going to be a large enough usage case. A standard business decision process where idealism takes a back position to shareholder interests (i.e. profit).

In short, I'm glad you agree with me about the reason that such a mode to not exist. You, I, and the rest of this board would be a tiny percentage of the market that Intel/AMD/etc. would be interested in supporting. Even all of us together could not form a large enough case to convince them to listen to us. It is not an ideal situation but it is where we are.
Post 04 Apr 2014, 07:32
View user's profile Send private message Visit poster's website Reply with quote
neville



Joined: 13 Jul 2008
Posts: 507
Location: New Zealand
neville 05 Apr 2014, 03:28
I'm glad that by contradicting yourself on point 4, and some fancy footwork on several other points (see below), you've been able to agree with me on most things except the complexity issue Laughing

So I'm also glad that you've agreed with me that you earlier reference to "community" actually meant the big corporate players, and not "hobby" or other communities, like we fasmers for example, with little or no financial clout.
That is an important distinction, and your initial statement could be construed as somewhat misleading on that point Wink

But I'm indifferent that you haven't been able to take that last little step and concede that a flat memory mode of whatever bus width is a uesful mode to have. Worse still you seem to think that I agree with you on this Sad

So because you're still obviously confused, I'll make another attempt to help you see where I'm coming from, if you want to of course!

For further reassurance on Point 1, have a look at the Contributor's Lists on any of a huge plethora of industry-standard specifications such as busses, peripheral interfaces like USB, northbidges, southbridges etc. You will see certain names cropping up again and again...

On Point 2, ask yourself what the "historical" reasons were. "Historical" by itself means little in this context as a justification. The irony is that Intel were ahead of the game in the early 8-bit days, when their main rival, Motorola, were stuck in the notion of memory-mapped I/O. Intel on the other hand recognised that a memory map without holes was much more desirable and so implemented separate memory and I/O address spaces, using an n+1th address line called IO/!M. Then later they almost ruined it all...

On Point 3, realise that "unsecurable" is NOT a synonym for "without paging or protection". Another misleading juxtaposition here? Many levels of security can be implemented in many different ways, none of which require either paging or what we know as "Protected Mode". Also, the more complex the environment, the harder it is to get the code right.

On Point 4, realise that "process isolation" can also be implemented in a variety of ways. Also a "careless" coder is an incompetent coder who will soon be weeded out if his code causes security issues for the OS or other processes.

On Point 5, now setting oneself up to form a new company to design, mass produce, and market a new CPU which is 100% X86-64 compatible but also capable of 64-bit flat real mode is just crazy pie-in-the-sky stuff, so it's not even worth a serious mention. Assuming it was serious.... Smile

On Point 6, oh, you seem pretty well-informed on that!

_________________
FAMOS - the first memory operating system
Post 05 Apr 2014, 03:28
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20298
Location: In your JS exploiting you and your system
revolution 05 Apr 2014, 03:59
It is not me, or the rest of this board, you need to convince. Talk to Intel/AMD about it. As for point 6, profit is going to the the most poignant issue to discuss with Intel/AMD. If you can't satisfy their desire for that then you will find it a very tough sell. If you can convince them to support such a mode then my hat is off to you for achieving such a thing against the odds.

As for point 4, sure, I often contradict myself, but in this case I was just a bit sloppy in my description. "Proper" (heh, oh I like those scare quotes) multitasking and multiuser OSes need process isolation to prevent the users getting upset because of coder A. But "improper" OSes can also offer multitasking and mutiuser but with a much reduced confidence or reliability. No major player is going to actively want to allow coder A to compromise the entire system, hence the lack of significant usage case for such a mode. Only hobbyist devs with little time and money available desire such modes but that market is tiny.

In a somewhat bizarre turn of events, with the inclusion of RIP relative addressing a non-paged multi-app OS would now be easier to implement in a flat model. But at the same time the availability of such a mode was removed. Razz Kind of like AMD sticking the finger to hobbyist devs and saying "just teasing ya".
Post 05 Apr 2014, 03:59
View user's profile Send private message Visit poster's website Reply with quote
sid123



Joined: 30 Jul 2013
Posts: 339
Location: Asia, Singapore
sid123 05 Apr 2014, 05:17
I am seriously (no sarcasm involved) going to ask Intel/AMD about this. Smile
Post 05 Apr 2014, 05:17
View user's profile Send private message Reply with quote
neville



Joined: 13 Jul 2008
Posts: 507
Location: New Zealand
neville 05 Apr 2014, 06:51
revolution wrote:
In a somewhat bizarre turn of events, with the inclusion of RIP relative addressing a non-paged multi-app OS would now be easier to implement in a flat model. But at the same time the availability of such a mode was removed. Razz Kind of like AMD sticking the finger to hobbyist devs and saying "just teasing ya".
Exactly. We are approaching the same wavelength! Shocked

@sid123 Well we can promise them a market size of least 2 customers Cool

_________________
FAMOS - the first memory operating system
Post 05 Apr 2014, 06:51
View user's profile Send private message Visit poster's website Reply with quote
timryazthunder



Joined: 18 Jan 2014
Posts: 44
timryazthunder 09 Apr 2014, 17:35
It should be called "Complete Real Mode"
Post 09 Apr 2014, 17:35
View user's profile Send private message Reply with quote
sid123



Joined: 30 Jul 2013
Posts: 339
Location: Asia, Singapore
sid123 12 Apr 2014, 07:54
timryazthunder wrote:
It should be called "Complete Real Mode"

Why? IMO Complete Real Mode would mean the 8086 Real Mode without any extended registers and instructions.

_________________
"Those who can make you believe in absurdities can make you commit atrocities" -- Voltaire https://github.com/Benderx2/R3X
XD
Post 12 Apr 2014, 07:54
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.