Since I have very little non-trivial code blocks which have actualy 2 different call sites, I am back on the coding hygiene of huge code paths.

In order to manage data and regs spiling on the stack on those paths, I wonder which path I should take: waste a register as a dynamic frame pointer (aka a 2nd level stack), or go full static stack mapping (data will be sparse on the stack frame).

Guys, do you have some experience on that, any feedback?

Without any sample code to examine it is hard to know what is being asked.

How is the stack currently handled? Does it use RBP to access locals, or RSP, or other register(s)? Are locals named, or is it a PUSH-PUSH-PUSH...POP-POP-POP pattern everywhere?

As of 'revolution', I barely understand the point, but I'll share something I notice that modern compilers do, like 'gcc', in hope that it may help you decide.
That thing is: modern C compilers loves stack and the stack pointer register so much. Almost everything defined as uninitialized variables on C code, is pushed/allocated into stack, with relation to 'rsp' register. This, whenever all non-volatile registers are already in use. Otherwise registers are used instead, to be C code variables.
I'm talking about code with '-O3' optimization.
This makes sense, because, one of its advantages is that you can completely remove an uninitialized data section (.bss), saving space and memory usage, because stack is always (and already) mapped.

In my personal experience, everytime you do access memory (for reading or for writing) the speed is compromised a little, because memory is slower with relation to the processor. This I've measured myself, and one can also do it, because I've published a tool (here, somewhere) to measure it.
As a general rule, I mostly keep things on non-volatile (callee-saved) registers as much as possible, using memory for less accessed stuff, and when there's no more registers left. Even reusing those registers whenever I can. And I mostly do it the simplest way: using stack and rsp register only (no stack frames with rbp), and always keep guarding stack alignment myself.

As stated, please share some code with examples for us, so the point will be clear.

Important: everything stated by me here, I've done on Linux only! It was long ago I've tested anything on Windows, so, if you're using on it, you may need to do some tests.

In time:
¹ there's nothing wrong to use a stack frame; the only point is, one less non-volatile register (rbp) at disposal.
² many approaches can be used: it all depends, if the code involved is time critical, or called many times. These are critical situations.

Hope it helps,

If I understood well, what you said about gcc -O3: they brutally and statically book room on the stack for all variables of the code paths of a function, which makes stack frame usage be memory sparse depending on branching.


What about the usage of a dynamic frame pointer usage (could be RBP on x86_64)? Namely function code blocks would be some sort of "sub-functions" which would use the stack frame as some sort of "sub-stack", if you see what I mean.

That way, usage of the stack frame should be less sparse than with a brutal static allocation (basically, should reduce the number of used cache lines at the price of locking a register as a "sub-stack" pointer).

gcc is smarter than brute force. It tracks usage lifetimes and reuses slots when variables go out of scope.

@revolution

Oh, that's what I would expect from a compiler. That's the reason why beating a compiler on complex and long code path is mostly not supposed to happen, and that' s why nowdays, not writting 'one-compilation-unit' compatible code for any binary(exe or dl) is seriously obsolete with all the ram and super fast storage we have (C "static" is so much important with pre-processor based namespaces)

So this is not the brutal static way we talked about. Well, not the brutal way I thought about.

So, I am back at this 'sub-stack' pointer, because this is a way to 'reuse' those 'stack frame slots' taking into account runtime inner branching. In this very case, it will probably do a better job at "packing in cache lines" those stack frame slots than the compiler way you did describe, that in many cases (but not all).

Currently, to track all that in the code, I use vim folds to annotate "roughly" verbosely the code (per branch and per dominator).

Well, I still can, and with a broad margin! Compiler been good at one or two things doesn't mean it does all stuff right. Don't be fooled! Even nowadays, I see a lot of strangeness coming from 'gcc', but I must point out that is less than when, let me say, 2 years ago. So, it is been improved. Also important, one must use the correct C terms and words, in the correct sequence, to obtain the best code, such thing most people don't do. And use optimizations to its maximum, another thing most people don't do. So, if you'll need to follow a strict, precise line on C to get the juice out of its code, why not assembly, then? Where the optimization level is only the programmer himself?

That's the reason I completely quit programming in C, because I look to the generated code, and felt ashamed that such an output was my code!

And, in my opinion, a well trained human assembly programmer will always win, because one has the most flexible language at its disposal. And one programming in assembly isn't letting compiler do its work instead.
And I, as a human assembly language programmer, just keep an eye on compiler, to learn its best ideas, as a second thought to my own ideas.

@jessé

I agree 100%.

But from a practical point of view, how do you track your register usage and spilled data/regs in complex and deep code paths?

As I said, on my side, I annotate blocks with a lot of information in vim folds (roughly, I actually hardly do it properly anytime): for each entry point, I maintain a register state & spilled data/regs for each dominator ("calling code"). Well, usually, I factor this information, as often I have a (all entries/all dominators) register/stack frame state.

I don't think there is any way around that... or am I severely wrong?

I don't have an official and defined way of doing it. I'm quite comfortable on handling things mostly by numbers and registers.
But, in general, I comment things, like: 'mov r12, rax ; r12 = screen' and then, somewhere: 'mov rsi, r12 ; r12 = obtained screen ID', and then, when reusing it: 'mov r12, rax ; r12 = context'.
This is enough to me. I still can read and understand an "extensive" code I did almost 15 years ago (8k lines, approx.), and uderstand all ideas I put in there. Even myself not even following that previous styling anymore.
I also like the simple idea of creating a comment header upon a function, or even an inline procedure, with an objective description of what is being done, and what is what there.

I'm recently testing using an "less pure assembly way", might I say, when I define a name for the holder of a parameter, instead of commenting, with quite good results, like:


That's what I can remember I always do and did...

I am trying to code a big ABI function. Some parts of this function are already at the 4th depth level: the 'working' register state ain't small at all. That why fine-grained register spilling is something that matters, since I need to make some room in the register state without brutally moving data like the ABI would do (because of the split of call-preserved/tmp registers).

Verbose register/stack frame state annotations are mandatory not only for my future self, but even for my now self. Vim folds help a lot.

What I would like is to keep the usage of the ABI function call convention for only non-trivial functions with at least 2 call sites. That said, while in the middle of the development cycle (I don't know yet how the code structure will turn out to be in the end), the flexibility of the ABI is more than welcome. I am sorta torn.

That's why I am trying to devise some coding method for those big functions: namely be more efficient at writting them, reducing the cost of modifications while in the middle of the development cycle.

I am looking for that, if I am not satisfied, I'll jump back to abusive ABI function usage while in the development cycle, which I'll fused once the code structure does settle down.

This is an "experiment" and I am welcoming the experience of others on that matter.