flat assembler
Message board for the users of flat assembler.

Index > Linux > Is my Linux system broken?

Author
Thread Post new topic Reply to topic
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20421
Location: In your JS exploiting you and your system
revolution 26 Sep 2023, 09:24
I found a problem with using the vsyscall interface. It corrupts calls that need six arguments. AFAICT it affects the following functions: mmap2, futex, sendto, recvfrom, mbind, splice, move_pages, epoll_pwait, process_vm_readv, process_vm_writev. Those all regure six arguments.

The reason for this is because the kernel uses syscall (on 64-bit machines) to do the underlying transition to kernel space. So RCX is overwritten by syscall. To counter that the vDSO copies ecx into ebp. And those paying attention will notice that ebp is also the sixth parameter for kernel calls. Hence the above functions are broken.

The vDSO code does not check for these broken functions and simply blindly jumps to the kernel (probaby because it is "faster"). But it isn't faster if it breaks code! There is no point going faster if you run in the wrong direction. Razz

Some code to show the equivalent broken vDSO code.
Code:
; instead of the less performant int 0x80
; it is possible to use syscall in 32-bit code

; to comply with the Linux syscall code
; the stack needs to be setup correctly
; with the return address and ecx edx ebp pushed
; upon return ecx edx ebp are poped,
; and execution is returned to the caller

; note that it is not possible to call a function
; that takes six arguments because ebp is copied
; from ecx and ecx is overwritten by syscall
; this affects mmap2, futex, sendto, recvfrom, 
; mbind, splice, move_pages, epoll_pwait,
; process_vm_readv, process_vm_writev

format elf executable

; use the standard 32-bit function numbers
SYS32_write             = 4
SYS32_exit_group        = 252
STDOUT_FILENO           = 1

entry $
        mov     eax,SYS32_write
        mov     ebx,STDOUT_FILENO
        mov     ecx,text
        mov     edx,len
        call    syscall32
        mov     eax,SYS32_exit_group
        xor     ebx,ebx
        call    syscall32

syscall32:
        push    ecx edx ebp
        mov     ebp,ecx         ; BROKEN: the sixth parameter is corrupted!
        syscall

text    db      'Hello, World',10
len     = $ - text    
Post 26 Sep 2023, 09:24
View user's profile Send private message Visit poster's website Reply with quote
Furs



Joined: 04 Mar 2016
Posts: 2543
Furs 26 Sep 2023, 13:34
Interesting, did you file bug reports about them yet?
Post 26 Sep 2023, 13:34
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20421
Location: In your JS exploiting you and your system
revolution 26 Sep 2023, 14:27
Furs wrote:
did you file bug reports about them yet?
No. I have no idea who to report to. Plus my kernel is five years old so it might already be fixed, or changed how it works or something.

I just made a macro to use int 0x80 for functions that need six arguments, so it is easy to mitigate.
Post 26 Sep 2023, 14:27
View user's profile Send private message Visit poster's website Reply with quote
weyowe



Joined: 04 Feb 2023
Posts: 6
weyowe 27 Sep 2023, 06:13
revolution wrote:
Plus my kernel is five years old so it might already be fixed, or changed how it works or something.

This sounds like an interesting issue, so I decided to check the source. It seems, the modern kernel always overwrites ebp for both sysenter and syscall:
Code:
; On Intel
push ebp
mov ebp,esp
sysenter

; On AMD
push ebp
mov ebp,ecx
syscall
    


Then kernel entry code takes the sixth argument from the saved ebp value in the stack.

However, the vDSO-kernel interface can change between different kernel versions - comments in the source files mention that in 2015 it caused problems to android developers who carelessly hardcoded sysenters in their libc - so it is hard to tell what exactly is happening on an older system.

But I find it hard to believe such a serious bug can bypass final tests. Usually, libc opts for vDSO wherever it can, and mmap is a basis for memory allocation and library loading. So, assuming the libc is not very old, this issue would render the entire 32-bit subsystem unusable. Are you sure you are not missing something?
Post 27 Sep 2023, 06:13
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20421
Location: In your JS exploiting you and your system
revolution 27 Sep 2023, 06:23
mmap is fine, It affects mmap2.
Post 27 Sep 2023, 06:23
View user's profile Send private message Visit poster's website Reply with quote
weyowe



Joined: 04 Feb 2023
Posts: 6
weyowe 27 Sep 2023, 07:10
As far as I know, the regular "mmap" (in kernel code it is referred as old_mmap) was introduced in early kernel versions at the time when it was impossible to pass 6 parameters to the kernel. Nowadays, it only exists for compatibility reasons and is not really used anywhere. So, mmap2 is the real mmap which is used by libc. Or, at least, I've always thought so.
Post 27 Sep 2023, 07:10
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20421
Location: In your JS exploiting you and your system
revolution 27 Sep 2023, 07:17
I made PoC code.
Code:
;       for SYS in syscall32 int0x80 syscall32 int0x80 ; do fasm -d SYS=$SYS vsyscall32.asm >/dev/null && ./vsyscall32 ; echo ; done

format elf executable

SYS32_write             = 4
SYS32_open              = 5
SYS32_mmap2             = 192
SYS32_exit_group        = 252
STDOUT_FILENO           = 1
O_RDONLY                = 00o
MIN_ERROR               = -4095
MMAP_PROT_READ          = 0x1
MMAP_MAP_PRIVATE        = 0x2

arg_address             = 0
arg_length              = 4
arg_access              = MMAP_PROT_READ
arg_flags               = MMAP_MAP_PRIVATE
arg_offset              = 0

entry $
        mov     eax,SYS32_open
        mov     ebx,fname
        mov     ecx,O_RDONLY
        xor     edx,edx         ; mode
        call    SYS
        cmp     eax,MIN_ERROR
        jae     .done
        ;
        mov     edi,eax
        mov     eax,SYS32_mmap2
        mov     ebx,arg_address
        mov     ecx,arg_length
        mov     edx,arg_access
        mov     esi,arg_flags
        mov     ebp,arg_offset
        call    SYS
        cmp     eax,MIN_ERROR
        jae     .done
        ;
        mov     ecx,eax
        mov     eax,SYS32_write
        mov     ebx,STDOUT_FILENO
        mov     edx,arg_length
        call    SYS
        cmp     eax,MIN_ERROR
        jb      .done
        ;
        mov     eax,SYS32_write
        mov     ebx,STDOUT_FILENO
        mov     ecx,failed_text
        mov     edx,failed_length
        call    SYS
    .done:
        mov     eax,SYS32_exit_group
        xor     ebx,ebx
        call    SYS

syscall32:
        push    ecx edx ebp
        mov     ebp,ecx
        syscall

int0x80:
        int     0x80
        ret

fname           db      '/proc/self/exe',0
failed_text     db      'Failed'
failed_length   =       $ - failed_text    
The output for me is:
Code:
~ for SYS in syscall32 int0x80 syscall32 int0x80 ; do fasm -d SYS=$SYS vsyscall32.asm >/dev/null && ./vsyscall32 ; echo ; done
Failed
ELF
Failed
ELF    
Post 27 Sep 2023, 07:17
View user's profile Send private message Visit poster's website Reply with quote
weyowe



Joined: 04 Feb 2023
Posts: 6
weyowe 27 Sep 2023, 07:43
Well, it is strange. I don't see anything wrong here, and both variants run without troubles on my system. It still makes me wonder what is going on inside libc. Maybe it is just too old and is not aware of vDSO, or this problem was a known issue, and it has some workarounds.
Post 27 Sep 2023, 07:43
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20421
Location: In your JS exploiting you and your system
revolution 30 Sep 2023, 10:17
So if others have no problem with 32-bit syscall then my system is broken? Crying or Very sad

Maybe my kernel is borked. I also see a problem with the MSR interface.
Code:
~ for OFF in {0..59} ; do for REG in 130 ; do ADDR=$(( (1 << OFF) | REG )) ; VAL=$(sudo dd bs=8 if=/dev/cpu/0/msr count=1 status=none skip=$ADDR 2>/dev/null | xxd -p ; test ${PIPESTATUS[0]} -eq 0) && printf "%16x = $VAL\n" $ADDR ; done ; done
              82 = 203b030200000000
              82 = 203b030200000000
        20000082 = 203b030200000000
        40000082 = 203b030200000000
... <snip duplicates> ....
 400000000000082 = 203b030200000000
 800000000000082 = 203b030200000000    
All MSR addresses are masked to 29 bits length. So I can't read, for example, MSR 0xc000_0082 because it aliases to 0x82 instead.
Post 30 Sep 2023, 10:17
View user's profile Send private message Visit poster's website Reply with quote
Furs



Joined: 04 Mar 2016
Posts: 2543
Furs 30 Sep 2023, 17:07
There might be a kernel config you're using. Are you using vanilla upstream kernel, a distro kernel, a custom config kernel…?
Post 30 Sep 2023, 17:07
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20421
Location: In your JS exploiting you and your system
revolution 30 Sep 2023, 19:32
Furs wrote:
There might be a kernel config you're using. Are you using vanilla upstream kernel, a distro kernel, a custom config kernel…?
I thought I was using stock Mint from about 5 years ago. IT are not forthcoming about how they validate and authorise the kernels. Perhaps they also make some "helpful" changes, "for security".

A lot of the system-related MSRs are above 2^29. Now I wonder if there is some "bonus" code in my kernel image.

So anyway, now I have learned something valuable. My system is broken, and it might be deliberate. Crying or Very sad
Post 30 Sep 2023, 19:32
View user's profile Send private message Visit poster's website Reply with quote
Furs



Joined: 04 Mar 2016
Posts: 2543
Furs 01 Oct 2023, 14:19
You can usually find the config where the kernel image is installed, e.g. /boot/config-<version>, maybe try compare with vanilla kernel's config?

I'm actually interested myself. Smile
Post 01 Oct 2023, 14:19
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20421
Location: In your JS exploiting you and your system
revolution 02 Oct 2023, 11:01
I'm not sure sure what I am looking for.
Code:
/boot grep -i syscall config-$(uname -r)
CONFIG_HAVE_ARCH_AUDITSYSCALL=y
CONFIG_AUDITSYSCALL=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_SGETMASK_SYSCALL=y
CONFIG_SYSFS_SYSCALL=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_BPF_SYSCALL=y
CONFIG_ADVISE_SYSCALLS=y
CONFIG_X86_VSYSCALL_EMULATION=y
# CONFIG_LEGACY_VSYSCALL_NATIVE is not set
CONFIG_LEGACY_VSYSCALL_EMULATE=y
# CONFIG_LEGACY_VSYSCALL_NONE is not set
CONFIG_MODIFY_LDT_SYSCALL=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_FTRACE_SYSCALLS=y

/boot grep -i msr config-$(uname -r)
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_X86_MSR=m
CONFIG_SCSI_ARCMSR=m

/boot    
If the source has been modified, or the binaries have been patched, or an extra module has been included, or any of about a million other things has changed, then that can change the behaviour of the system, right? Does the config file validate that all the kernel stuff is pristine and unadulterated?
Post 02 Oct 2023, 11:01
View user's profile Send private message Visit poster's website Reply with quote
Furs



Joined: 04 Mar 2016
Posts: 2543
Furs 02 Oct 2023, 14:05
The config simply says what config it was compiled with, there's no guarantee it's how it works since you can change the source code to anything you want before building, but what's the point of them lying about it? Unless you got the kernel from a fishy place, obviously. But in that case you got bigger problems to worry about. Razz

I'm not sure if the config necessarily has "syscall" in it. I know it's huge, but try to get the same upstream kernel (not distro, same version I mean), test it in a VM and see if it has the same problem. If it doesn't, try compare their configs and rule them out with bisection.

Building the kernel is pretty easy since there's no crazy dependencies, despite its large size, just do it in tmpfs to not pollute your hard drive.

Obviously only if you have the time for it.

Actually before trying configs, you could try to get the source of your distro's kernel (exact ver, they should supply it in their packages), and compile it with the upstream vanilla kernel's config. See if it also has the problem. If it does, it's a custom patch in your distro's kernel that is causing this. They usually have many patches above the upstream one, that is provided in the sources. Once you found the culprit you can start bisecting the patches with git bisect.


(BTW, I have the same configs with that grep, that's why I'm saying it)
Post 02 Oct 2023, 14:05
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20421
Location: In your JS exploiting you and your system
revolution 02 Oct 2023, 14:25
The closest I had to a VM on this Linux box was DOSBox which I got permission to install for the 512 byte contest. It got removed as soon as that was finished. Getting permission to install a "real" VM for the reason of detecting IT meddling? Hmmm, methinks the chances are slim.

If the kernels are reproducible builds, or simply copied from an install medium (ISO file?), perhaps I can do a simple diff? I'd need to find the ISO image and mount it though. I have to include IT for that also. Mounting unknown stuff requires an admin sign-off.

What else is there? md5sum? Only a matching result means anything, a mismatch wouldn't give any clues of value.
Post 02 Oct 2023, 14:25
View user's profile Send private message Visit poster's website Reply with quote
Furs



Joined: 04 Mar 2016
Posts: 2543
Furs 03 Oct 2023, 13:44
I'm not saying it's malicious, it's probably just someone messing a config up. Hashes won't tell you much except that it's "different", which it very likely is, most distros ship patched kernels (usually for security backports). So there's nothing to be alarmed about if hash doesn't match. It's expected.
Post 03 Oct 2023, 13:44
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.