flat assembler
Message board for the users of flat assembler.
 Home   FAQ   Search   Register 
 Profile   Log in to check your private messages   Log in 
flat assembler > Linux > Linux ABI stack alignment

Goto page 1, 2, 3  Next
Author
Thread Post new topic Reply to topic
fasmnewbie



Joined: 01 Mar 2011
Posts: 396
Linux ABI stack alignment
After making some tests, I found that the kernel consistently allocates stack that are aligned to 16 byte boundary for executable ELF64. But it is not true in ELF64 object, which is aligned to natural boundary ( 8 byte ) during entry.

I have no way of telling whether this is intended by design, by random or unattended behavior since I have only one Linux machine to run my tests. I hope others can help provide alternative results from their own OS in order to confirm this.

Thanks in advance.
Post 12 Apr 2017, 12:44
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 14794
Location: Lost in translation
I don't think this is something that should be measured, because what you see could easily be an artefact of coincidence.

I hate to point out the obvious but either the source code or the documentation, or both, should tell you what the alignment is expected to be.
Post 12 Apr 2017, 12:48
View user's profile Send private message Visit poster's website Reply with quote
fasmnewbie



Joined: 01 Mar 2011
Posts: 396

revolution wrote:
I don't think this is something that should be measured, because what you see could easily be an artefact of coincidence.

I hate to point out the obvious but either the source code or the documentation, or both, should tell you what the alignment is expected to be.



There is significant differences for both formats when dealing with alignment for the stack. What I am actually interested in knowing is the kernel's stack allocation policy. I don't think there're documented anywhere (or perhaps I missed the documentation entirely).
Post 12 Apr 2017, 12:54
View user's profile Send private message Visit poster's website Reply with quote
fasmnewbie



Joined: 01 Mar 2011
Posts: 396

Code:
format ELF64 executable 3

entry $

;The stack is consistently aligned to 16 by default here
;Now you can safely access any syscalls with guaranteed alignment
;no external alignment needed. Any attempt to align will break the whole alignment ecosystem.

;exit




Code:
format ELF64
public something

something:

;The stack is NOT GUARANTEED to be aligned to 16 by default here
;Now we're not sure anymore. Should we aligned the stack externally b4 using syscalls / C library? 

;exit



At least on my machine, this behaviour is consistent. But I don't know how they'd behave on other distros / PCs. This is where I need the confirmation, because this behavior is not documented anywhere

I hope this pseudo makes my question clearer. Thanks
Post 12 Apr 2017, 13:16
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 14794
Location: Lost in translation
Linux doesn't require 16-byte alignment. Perhaps you are confusing things with Windows fastcall convention.
Post 12 Apr 2017, 13:17
View user's profile Send private message Visit poster's website Reply with quote
fasmnewbie



Joined: 01 Mar 2011
Posts: 396

revolution wrote:
Linux doesn't require 16-byte alignment. Perhaps you are confusing things with Windows fastcall convention.



It doesn't? Are you sure my friend?
Post 12 Apr 2017, 13:22
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 14794
Location: Lost in translation
Post 12 Apr 2017, 13:25
View user's profile Send private message Visit poster's website Reply with quote
fasmnewbie



Joined: 01 Mar 2011
Posts: 396
How about this, then? It's under "Stack Frame" sub-heading.

http://chamilo2.grenet.fr/inp/courses/ENSIMAG3MM1LDB/document/doc_abi_ia64.pdf
Post 12 Apr 2017, 13:27
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 14794
Location: Lost in translation
Okay, maybe you are right. There often seems to be conflicting information about Linux.

In that case I would hesitate to always rely upon the test results to "confirm" that the stack is, or isn't, aligned correctly at the application entry point. Instead, just add the appropriate and rsp,-16 once as the first instruction and then you don't have to care about it again.
Post 12 Apr 2017, 13:44
View user's profile Send private message Visit poster's website Reply with quote
fasmnewbie



Joined: 01 Mar 2011
Posts: 396

revolution wrote:
Okay, maybe you are right. There often seems to be conflicting information about Linux.

In that case I would hesitate to always rely upon the test results to "confirm" that the stack is, or isn't, aligned correctly at the application entry point. Instead, just add the appropriate and rsp,-16 once as the first instruction and then you don't have to care about it again.



You're not entirely wrong on this either. I've experimented with some syscalls and they do work on unaligned stack, giving us such wrong impressions that Linux ABI don't need strict stack alignments.
Post 12 Apr 2017, 13:50
View user's profile Send private message Visit poster's website Reply with quote
Furs



Joined: 04 Mar 2016
Posts: 359
revolution is half-right. The Linux Kernel does not require stack alignment. i.e. syscalls don't.

However, shared libraries do, since they follow the stupid AMD ABI.

If you only use the Linux Kernel (which is technically "Linux", libraries aren't), then you don't need alignment of stack. If you call into GNU libraries or whatever you'll need alignment though.
Post 13 Apr 2017, 11:17
View user's profile Send private message Reply with quote
fasmnewbie



Joined: 01 Mar 2011
Posts: 396

Furs wrote:
revolution is half-right. The Linux Kernel does not require stack alignment. i.e. syscalls don't.

However, shared libraries do, since they follow the stupid AMD ABI.

If you only use the Linux Kernel (which is technically "Linux", libraries aren't), then you don't need alignment of stack. If you call into GNU libraries or whatever you'll need alignment though.



I think the issue is not yet fully resolved. Follow the discussions here:

https://patchwork.kernel.org/patch/9507697/

I can't follow the discussions very well because those guys (including Linus) talk in C. Bleeds my eyes.

But my take on this is to just follow the ABI alignment requirement to avoid future problems. I have this hunch that the alignment requirement is actually meant for syscalls that make use of SSE instructions. I don't see any other practical reasons for enforcing aligned stack other than SSE. The alignment is not actually for the function's code, but for use by the SSE save and restore inside the red zone of one particular syscall. 128 bytes is enough for XMM0 and XMM7. They are the most likely candidate for such alignment or redzone population.

So the fact that many 64-bit code can safely disregard the alignment requirements is because, for backward compatibility, the syscall have rarely used SSE instructions / registers so far or at least they don't bother saving the SSE states.

But there's no guarantee in the future. If UNIX or Linux don't come clean on this as soon as possible, we could be witnessing massive restructuring of userland codes that have complete disregard for such alignment requirement, including my own, once the movdqa kicks in from any of the syscall.
Post 13 Apr 2017, 14:41
View user's profile Send private message Visit poster's website Reply with quote
Furs



Joined: 04 Mar 2016
Posts: 359
But Linux kernel is fully backwards compatible and they never introduce breaking changes. I don't understand your worry though? If the specification says it doesn't require 16-byte alignment, then it will always work without 16-byte alignment, unless they break the spec (which they won't, like I said, Linus values backwards compatibility; his words "we never break userland")

As a matter of fact they had a bug suggestion on GCC to allow 8-byte aligned stack for the kernel, to not waste space: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53383

Yes, the kernel barely uses SSE, and when it does it aligns the stack properly by itself. If in the future it will use more SSE, it will align the stack itself, so you don't have to worry about it when using syscall.

Then again, who's to say it won't use AVX or AVX512 in the future instead of SSE? (to be honest, they probably already do). The kernel is known to use the latest instruction sets, anyway, quite early might I add. Are you going to align the stack to 64-bytes? What if they add 128-byte vectors? See where I'm going?

I don't think you have to worry when doing syscalls, unless I misunderstand your situation? If the kernel will use vectors, it will align the stack itself, like a sane ABI.
Post 13 Apr 2017, 15:17
View user's profile Send private message Reply with quote
fasmnewbie



Joined: 01 Mar 2011
Posts: 396

Furs wrote:
But Linux kernel is fully backwards compatible and they never introduce breaking changes. I don't understand your worry though? If the specification says it doesn't require 16-byte alignment, then it will always work without 16-byte alignment, unless they break the spec (which they won't, like I said, Linus values backwards compatibility; his words "we never break userland")

As a matter of fact they had a bug suggestion on GCC to allow 8-byte aligned stack for the kernel, to not waste space: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53383

Yes, the kernel barely uses SSE, and when it does it aligns the stack properly by itself. If in the future it will use more SSE, it will align the stack itself, so you don't have to worry about it when using syscall.

Then again, who's to say it won't use AVX or AVX512 in the future instead of SSE? (to be honest, they probably already do). The kernel is known to use the latest instruction sets, anyway, quite early might I add. Are you going to align the stack to 64-bytes? What if they add 128-byte vectors? See where I'm going?

I don't think you have to worry when doing syscalls, unless I misunderstand your situation? If the kernel will use vectors, it will align the stack itself, like a sane ABI.



I am more concern with Linux kernel inconsistency regarding the stack allocation, hence my first post. That first stack allocation will determine my stack alignment policy for the next libraries / routines.

IF, by default Linux allocates the stack aligned to 16, then the rest would just follow suit to maintain such alignment ecosystem. IF, Linux allocates it using random alignment (8 or 16), then I have to determine the correct adjustment required for the next routines.

That's the central issue here, and I need help confirming both cases to determine its consistency. So far, both format apply different alignments, at least on my distro.

Thanks.
Post 13 Apr 2017, 16:02
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 14794
Location: Lost in translation
If you need some particular alignment then do it yourself. and rsp,-16: done. Just assume the alignment from the kernel is random, and make it what you need. It is only one single instruction, no big deal.
Post 13 Apr 2017, 16:11
View user's profile Send private message Visit poster's website Reply with quote
fasmnewbie



Joined: 01 Mar 2011
Posts: 396

revolution wrote:
If you need some particular alignment then do it yourself. and rsp,-16: done. Just assume the alignment from the kernel is random, and make it what you need. It is only one single instruction, no big deal.



and rsp,-16 = no ABI required.

That literally means you can disregard any calling convention out there. Is that what you're suggesting? My BASELIB is full of that Very Happy

https://board.flatassembler.net/topic.php?p=184548
Post 13 Apr 2017, 16:21
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 14794
Location: Lost in translation
Calling conventions are more than just the stack alignment. And in fact you would be complying with the convention (not disregarding it) if you align accordingly.
Post 13 Apr 2017, 16:24
View user's profile Send private message Visit poster's website Reply with quote
Furs



Joined: 04 Mar 2016
Posts: 359
Does the kernel even have callbacks?

If not, then you only need one and rsp, -16 at the beginning of your entire program, just to be safe, what's the big deal?

Anyway, I think it does align the stack to 16-byte before passing it to userland though. But then again, one instruction per program isn't going to kill it Wink

EDIT: I looked at the sources of libc (which is what usually runs before 'main' in C programs), and it seems it DOES align the stack (sorry I didn't read correctly before).

First it gets information from above the stack on entry to program (which is where Linux stores argc/argv/env vars), and then aligns the stack with and rsp, -16, so you should do it just in case.

The file is in sysdeps/x86_64/start.S of libc. (AT&T asm though, so beware)
Post 13 Apr 2017, 16:27
View user's profile Send private message Reply with quote
fasmnewbie



Joined: 01 Mar 2011
Posts: 396

revolution wrote:
Calling conventions are more than just the stack alignment. And in fact you would be complying with the convention (not disregarding it) if you align accordingly.



With and rsp,-16 you can completely disregard any conventions. Trust me, I did that to Win64 and Linux64 using the same identical source via BASELIB. I further extended the support to binaries (SO, DLL, LIB, OBJ, O etc) and not one ever emits any segfault. I managed to dodge the stack requirements for both ABIs via "and rsp,-16" alone.

"and rsp,-16" literally means no64- ABI required. It's the old 32-bit standard call calling convention Very Happy
Post 13 Apr 2017, 16:38
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 14794
Location: Lost in translation
If you need to call a system API and the system requires a particular stack alignment then you still have to comply to the convention. So using and rsp,-16 helps you to comply. If you are talking about your own internal code then you can do whatever you want, align to 1 if it pleases you, but don't try to call a system API like that, you'll need to place an and rsp,-16 to fix everything up and comply.
Post 13 Apr 2017, 16:42
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2, 3  Next

< Last Thread | Next Thread >

Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001-2005 phpBB Group.

Main index   Download   Documentation   Examples   Message board
Copyright © 2004-2016, Tomasz Grysztar.