[bug]Understanding the out of memory error. Piping into fasm

Index > Compiler Internals > [bug]Understanding the out of memory error. Piping into fasm

Author

Thread

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20303
Location: In your JS exploiting you and your system

revolution 12 Feb 2023, 14:51

I've often wanted to connect fasm to a pipe and send in the source without using an intermediate file.

Code:

echo 'db "hello world!",10' | fasm /dev/stdin /dev/stdout
flat assembler  version 1.73.08  (4034272 kilobytes memory)
error: out of memory.

We get an "out of memory" error.

Using stdout works fine.

Code:

fasm hello.asm /dev/stdout
flat assembler  version 1.73.08  (4036452 kilobytes memory)
hello world!
1 passes, 13 bytes.

In system.inc the lseek function checks the wrong value.

Code:

lseek:
        mov     ecx,edx
        xor     edx,edx
        mov     dl,al
        mov     eax,19
        int     0x80
        cmp     eax,-1    ; <--- lseek never returns -1
        je      file_error
        clc
;...

The error -1 is EPERM which is never a return value from lseek. The actual error when using a pipe is ESPIPE (-29) so fasm returns the position 0xffffffe3 with no error indicated to the caller.

Indeed many of the value return checks from int 0x80 are buggy, For example:

Code:

read:
        push    ecx edx esi edi ebp
        mov     eax,3
        xchg    ecx,edx
        int     0x80
        pop     ebp edi esi edx ecx
        test    eax,eax
        js      file_error ; <--- All values >2GB considered errors

A fix for all of these is:

Code:

;...
        int     0x80
        cmp     eax,-4095  ; <--- this is the same value use by glibc to detect errors.
        jae     file_error
;..

For the other matter of piping into fasm. Even if lseek and the other syscalls are fixed we still have to contentd with the code in preproce.inc.

Code:

preprocess_file:
        push    [memory_end]
        push    esi
        mov     al,2
        xor     edx,edx
        call    lseek
        push    eax
        xor     al,al
        xor     edx,edx
        call    lseek
        pop     ecx  ; <--- tries to get the length of the pipe, but will fail
        mov     edx,[memory_end]
        dec     edx
        mov     byte [edx],1Ah
        sub     edx,ecx
        jc      out_of_memory
        mov     esi,edx
        cmp     edx,edi
        jbe     out_of_memory
        mov     [memory_end],edx  ;  <--- memory is allocated only once
        call    read  ; <--- the only read done
;...

There is only one chance to get the the file length and adjust the memory settings. That can't be done with a pipe. The lseek calls would need to be eliminated. Set the memory to maximum (the edi value) and keep reading until zero bytes received. Then movsb into the high address, adjust pointers, and preprocess.

Last edited by revolution on 11 Jul 2023, 13:06; edited 2 times in total

12 Feb 2023, 14:51

FlierMate11

Joined: 13 Oct 2022
Posts: 94

FlierMate11 14 Feb 2023, 09:55

Thanks to your -4095 error value, it indirectly helps to solve my hexdump to catch file open error:

Like what you reported, my hexdump evaluate rax value as -1, but it didn't catch the open error.

Code:

      xor     rsi,rsi   ;O_RDONLY
      mov     rax,2     ; sys_create
      syscall
      cmp     rax,-1
      je      _err
      mov     dword [fd],eax

Until I saw this thread, then I changed it to:

Code:

...
...
      cmp     rax,-4095
      jae      _err
      mov     dword [fd],eax

...then only it able to catch the file open error, and avoid opening a non-existent file.

I tried "je _err" but it didn't work, have to use "jae _err", looks like the error code is above -4095.

14 Feb 2023, 09:55

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20303
Location: In your JS exploiting you and your system

revolution 14 Feb 2023, 10:06

The value returned is the negative value of the error (listed in "sys/errno.h"). So the returned value for an error can be anywhere from -4095 to -1.

The Linux kernel spec guarantees that all error numbers will be <=4095. So the negative will be >=-4095.

As mentioned above -1 == -EPERM is a permission fault error. Looking at the current errno.h I see that highest value is EHWPOISON = 133, well within the 4095 maximum range. If you only check for -1 then you are only catching permission fault errors.

14 Feb 2023, 10:06

FlierMate11

Joined: 13 Oct 2022
Posts: 94

FlierMate11 14 Feb 2023, 10:24

I think Linux manual is misleading, for example, for "creat(2)":

Quote:

Return Value
open() and creat() return the new file descriptor, or -1 if an error occurred (in which case, errno is set appropriately).

https://linux.die.net/man/2/creat

Followed the wrong guide.

14 Feb 2023, 10:24

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20303
Location: In your JS exploiting you and your system

revolution 14 Feb 2023, 10:34

That is the libc wrapper document. The kernel is different.

14 Feb 2023, 10:34

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20303
Location: In your JS exploiting you and your system

revolution 14 Feb 2023, 11:07

For reference here is the libc wrapper code of a random chosen syscall. All error values are negated and stored into the "errno" variable, and returns -1. The errno variable is a libc thing only. The kernel doesn't have it.

Code:

00000000004425d0 <__sysinfo>:
  4425d0:       b8 63 00 00 00          mov    eax,0x63
  4425d5:       0f 05                   syscall 
  4425d7:       48 3d 01 f0 ff ff       cmp    rax,0xfffffffffffff001 ; -4095
  4425dd:       0f 83 cd 1b 00 00       jae    4441b0 <__syscall_error>
  4425e3:       c3                      ret    

;...

00000000004441b0 <__syscall_error>:
  4441b0:       48 f7 d8                neg    rax

00000000004441b3 <__syscall_error_1>:
  4441b3:       64 89 04 25 d0 ff ff    mov    DWORD PTR fs:0xffffffffffffffd0,eax ; errno variable
  4441ba:       ff 
  4441bb:       48 83 c8 ff             or     rax,0xffffffffffffffff ; -1
  4441bf:       c3                      ret

14 Feb 2023, 11:07

FlierMate11

Joined: 13 Oct 2022
Posts: 94

FlierMate11 14 Feb 2023, 11:27

Thanks for the effort and informative explanation, @revolution.

I see it now.

revolution wrote:

That is the libc wrapper document. The kernel is different.

Next time where can I refer to for the kernel one?

14 Feb 2023, 11:27

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20303
Location: In your JS exploiting you and your system

revolution 14 Feb 2023, 11:31

The page you linked is good. Just ignore the errno and -1 return value stuff. Negate the error values shown and it's all good. Everything else is the same, including the argument ordering.

14 Feb 2023, 11:31

FlierMate11

Joined: 13 Oct 2022
Posts: 94

FlierMate11 14 Feb 2023, 11:36

revolution wrote:

The page you linked is good. Just ignore the errno and -1 return value stuff. Negate the error values shown and it's all good. Everything else is the same, including the argument ordering.

Thank you very much. Very Happy

14 Feb 2023, 11:36

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum