Why then wrote about the fact that the colon is optional.

I might be wrong, but I think the optional colon makes it compatible with NASM syntax. While allowing the colon solves some corner cases more gracefully.

There is. Such behaviour simplifies doing anti-debugging tricks for authors of shareware programs

I don't understand. How does a (bad) debugger modifying the target process with no reason, help the target process do anti-debugger tricks?

I don't know of any debugger that modifies the target without the user telling it to. Except for Furs' debugger, which Furs needs to delete, because it crashes at the drop of a hat, and makes random changes to things without being commanded to.

As for me, the word "optional" does not fit here. Here it is better to write "may be required".
When you can specify or not specify a colon in all combinations, this is optional.
When there are combinations that can be written only without a colon, others only with a colon, and still others both, it is better to write that the presence of a colon may be required.
times 20: or rcx, 0 With a colon in the middle, it's not so beautiful anymore.

The root of the "problem" is that or is also an arithmetic operator in fasm. So it tries to evaluate "20 or rcx".

I can certainly appreciate the desired aesthetic. Colon is associated with label in asm - injecting it into other syntax breaks that continuity. In fasmg I mainly use REPEAT or DUP. fasmg has many syntax colons, lol.

We can also handle exceptions and use RSP as a GPR:

If you really want, you can still generate an exception while executing "sub rsp, 1". For example by setting the trap flag.

Haha, I hope you are ready for an onslaught of exceptions, because that affects all instructions, including nop.

But if you ask for them, Windows will happily deliver them. Be prepared.

xor byte [rsp + 1], 3

If such normal stuff was the only thing happening all the time I guess there would be 0 CVE vulnerabilities in the world.

The fact you still haven't commented on the second example he gave (Windows developer btw) shows to me you have nothing to say to it because you know he's right.

Anyway, I would need a small ransom to actually intentionally trigger a page fault or IO error to prove you wrong, because it takes too much of my time for something that's obvious and silly for you to ignore.

Who is "he"? Which example?

"He" is Raymond Chen, from a blog link a few pages back, and the example was the code page failing to page in due to living on an ejected CD-ROM.

Granted, you should not misalign the stack, but the question is if a process can have an exception delivered without explicitly asking for it. Then revolution’s argument holds – if you expect an exception delivered, have your stack prepared, but otherwise, do as you like. Even then, the exception delivery will probably also work on a misaligned stack because the hardware does not really care about the alignment. It’s just some CRT/API functions that expect an aligned stack, usually because they move memory around using SSE, and it is questionable whether Windows uses these during exception delivery. Again, if you call these functions, you need to abide by their rules, but if you don’t, I guess you can get away with all kinds of abuse. And the paging-in example is a one-in-a-billion situation that you will likely never encounter during testing…

I know that Transport Tycoon Deluxe for Windows used to run on a misaligned stack (it pushed 16-bit segment selectors onto it in a few places), which seems to have worked back then. I don’t know which version of Windows it was targeted at; probably Win95. When I experimented with binary patching on Windows XP a bit a few years ago, it tripped up badly though, because random Windows API functions did not like it.

Ringding: RFL.AC

Oh, the RC article. Thanks Ringding. But the "paging-in example is a one-in-a-billion situation" is not correct. Let me explain:

I am happy to acknowledge that the article is perfectly 100% correct. I never said it wasn't. Raymond Chen is a smart guy and I have no reason to doubt the correctness of the article.

What is wrong here is the assumption that it is relevant. The reason I didn't comment specifically on anything there is because it isn't relevant. I stated the lack of relevancy earlier.

I think a lot of this is caused by misconceptions about how exceptions work in Windows.

Fact 1:
You won't get any exceptions delivered unless you ask for them, or if some library asks for them on your behalf.

Fact 2:
You only get exceptions that are relevant to the thread. Not random stuff that your code isn't directly involved with, or can't deal with.

Fact 3:
Exceptions are always delivered in response to the specific instructions that caused them, and not for any other reason. They aren't delivered at random times or to random threads. They are only delivered during execution of the specific instruction involved, not before and not after.

So from those facts I can explain in more detail why the article isn't relevant.

Firstly: The article is talking about values in the memory below RSP, which sub rsp, 1 isn't involved with.
Secondly: The article uses a mov reg, [mem] instruction, which can indeed generate an exception. sub rsp, 1 doesn't generate exceptions. Using the trap flag doesn't cause sub rsp, 1 to generate an exception, it is the trap flag that generates the exception.

Because of fact 3 above, mov reg, [mem] has the potential to be "exceptional" (hah!). But the code we were discussing originally is sub rsp, 1 which has no potential for "exceptional" behaviour. No exceptions are delivered by the kernel for sub rsp, 1, it has nothing to deliver (see fact 3). The kernel won't be triggered by sub rsp, 1 unless an exception is generated, which it isn't. There is no exception type listed as "instruction can't generate any exception".

If the kernel pages out your code to the swap file because some other process had gobbled up all the RAM, and now it wants to restart your code, what happens? Will your code get an exception? No it won't. It was the kernel that swapped it out, and it is the kernel's job to swap it back in. Your code can't do that (your code isn't even in memory yet), only the kernel can do that. So because of fact 2 above, you don't get an exception for that. Even if you did get an exception you couldn't do anything about it anyway, you didn't cause it, and you can't fix it, so there is no reason to deliver it (fact 2). And because of fact 3, it isn't related to any specific instruction, it is a system action not relevant to your code (fact 2).

What happens if the user moves the mouse? Will your code get an exception then? No it won't. Your code can't do interrupt processing, so the kernel won't deliver an exception to you. Because of fact 2 above, you won't even know about it (unless perhaps you are really clever with monitoring the TSC or something).

But won't the interrupts cause stuff to be stored on my stack? No they won't. A properly configured protected mode OS (which Windows is) sets up the ring-0 RSP value for interrupts to use. Your stack is never touched by any interrupt processing.

But the debugger is running in my process and it uses the stack, right? No it doesn't. A debugger runs as an independent process with it's own stack, register set and address space. It can't directly access your address space, it has to mediate all accesses via the kernel interface API.

But the debugger can change my stack? Yes it can. It can also change your code, or any other memory value, modify the register values, and kill the process. None of that is relevant to sub rsp, 1. Your code can't possibly predict the actions of a debugger, or the user controlling it. We have to assume the debugger is not broken (or evil), else we could never trust any code to do anything, ever.

So there you have it. Your app's RSP value does not need to be maintained to any particular value or range. Do whatever you want with it. Just be mindful about your exceptions and avoid the call/ret/push/pop stuff and everything will be fine. It is easy to write code that never generates exceptions, and/or is aware of possible problematic "exceptional" instructions, I showed this above with two example programs that use RSP as a GPR. There are no "time bombs" in those programs. They will always work fine. There is no place for exceptions to sneak in unexpectedly. Exceptions are not random and unpredictable, they can be perfectly characterised and understood. In the first example you can see that there are no instructions used in the loop that can generate an exception. In the second example, the one instruction that generates an exception is wrapped in code to restore the stack, and allow the exception to happen.

But why do I have to restore the stack to get an exception delivered, can't I just set RSP to a different memory region? In later versions Windows this isn't possible, it checks to see if RSP points to the memory region designated as the stack. This is a recent change in Windows that was made to "fix" some security problems. Older versions of Windows do allow it.

Furs understands this explanation now? Yes or no? If no, please explain. If yes, please acknowledge.

I am really enjoying this thread. I try hard to improve my explanation skills, which I know are poor. I hope I have explained things a bit better. Please feel free to ask more if something still isn't clear. I had trouble with the first person / second person / third person thing, I hope it isn't too confusing.

Ok. I see what you mean. If Windows can't deliver random exceptions/signals (in Unix land you can) then sub rsp, 1 might be safe by itself, but using your stack as a GPR is still unsafe if you do any sort of memory access (or misaligning it if you expect exception frame to be pushed on the stack).

Unless you're missing something. The debugger example, btw, is not strictly about debuggers, but the debugging API. There's some stupid DRM/anti-cheat stuff that will do that kind of hacks, but I guess you probably don't have to worry if it's not related to them.

If the memory has already been committed then you will not get any exceptions for it, even if it is paged out. The page-in / page-out operations are supposed to be completely transparent to an app. You have no control over them, so you are not expected to be able to know about them, or care about them.

Memory only marked as reserved, and then accessed by code will definitely generate an exception.

By way of illustration about how writing an app that never generates an exception is not difficult, just look at fasm.

There is no exception handler in fasm. And even if it installed one it would get no exceptions to handle.

How can we know this? Because when we run fasm it doesn't crash.

Broadly speaking there are only two main paths the kernel takes when an exception occurs:
1: Traverse the exception chain if it exists and allow the app the fix itself, or
2: Kill the app

There is no third option of ignoring the exception. All exceptions have to be dealt with in Windows. And since fasm doesn't have an exception chain the kernel will kill it if an exception ever occurred.

So each time you run fasm, you don't have to care if the system pages it in and out many times, it is all transparent, and doesn't create an exception. And fasm certainly reads a lot of memory, and still it doesn't ever create an exception. So reading memory is not a problem in many cases even if your stack is NULL.

So if it wanted to, fasm could run with esp pointed to nowhere, and somehow implement call/ret/push/pop in another way, and it would all work fine. Just make sure when calling the system APIs to give it the required real stack.

When we don't understand something we become fearful of it. We develop superstitions around it so as not to "anger the gods". We worry that we haven't covered all the possibilities.

I see this kind of behaviour happening around exceptions.

So I can see from that quote that I didn't explain things well enough to promote proper understanding.

While accessing memory can generate an exception, it only happens if you have a bad pointer, or the memory has not yet been committed. The latter is useful for reserving address space and later committing RAM when it is needed. But most apps don't use this feature, so it is rarely encountered.

If you have a bad pointer then you will know about it. Your app crashes, Windows displays the dreaded dialogue box telling your app failed and will be terminated. Or if you add some code to the exception chain then your app can fix itself or kill itself, whatever makes sense for it.

But either way, exceptions happen very rarely in most apps, and they happen never in apps like fasm.

Another common source of exceptions is div when the high {q|d}word of the numerator is greater than the denominator or the denominator is zero. You will also know about that one when it happens. Either crash, or fix yourself, you have to choose one of them.

So just to reiterate, accessing properly committed memory (which is generally all of normal memory in most apps) never generates an exception. Don't worry about exceptions, they won't ever happen, regardless of the disposition of the actual RAM. It might be in swap space, or not yet loaded from the exe file, it doesn't matter, it's committed, you will eventually get to access it without creating any exceptions.