flat assembler
Message board for the users of flat assembler.

flat assembler > Unix > Mach-O executables made with fasmg

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6862
Location: Kraków, Poland
I have finally been able to write some macros for fasmg that allow creation of Mach-O executables. I initially thought this could be similar to ELF, where creation of executable files is relatively simple and the main complexity is in a preparation of object files for linking. But now I feel that is is actually the opposite - the object Mach-O may be not that different from ELF, so the analogous macros may be not that hard to create. On the other hand, the Mach-O executable files contain a lot of nuance in the structures for loader and dynamic linker. Consider also that some of the structures and requirements were changing from one version of MacOS to another, and this really adds up to an unfriendly environment.

There are some challenges that still require a lot of work. I would welcome any help if any of you would like to experiment with this.

Let me present the macros I have prepared so far. First, a bunch of equates extracted from various header file. We are going to need many of them:
Code:
CPU_TYPE_ANY = -1 CPU_ARCH_ABI64 = 0x1000000 CPU_TYPE_VAX = 1 CPU_TYPE_ROMP = 2 CPU_TYPE_NS32032 = 4 CPU_TYPE_NS32332 = 5 CPU_TYPE_MC680x0 = 6 CPU_TYPE_I386 = 7 CPU_TYPE_X86_64 = CPU_TYPE_I386 or CPU_ARCH_ABI64 CPU_TYPE_MIPS = 8 CPU_TYPE_NS32532 = 9 CPU_TYPE_HPPA = 11 CPU_TYPE_ARM = 12 CPU_TYPE_MC88000 = 13 CPU_TYPE_SPARC = 14 CPU_TYPE_I860 = 15 CPU_TYPE_I860_LITTLE = 16 CPU_TYPE_RS6000 = 17 CPU_TYPE_MC98000 = 18 CPU_TYPE_POWERPC = 18 CPU_TYPE_POWERPC64 = CPU_TYPE_POWERPC or CPU_ARCH_ABI64 CPU_TYPE_VEO = 255 CPU_SUBTYPE_MASK = 0xff000000 CPU_SUBTYPE_LIB64 = 0x80000000 CPU_SUBTYPE_I386_ALL = 3 CPU_SUBTYPE_X86_64_ALL = CPU_SUBTYPE_I386_ALL CPU_SUBTYPE_386 = 3 CPU_SUBTYPE_486 = 4 CPU_SUBTYPE_486SX = 4 + 128 CPU_SUBTYPE_586 = 5 CPU_SUBTYPE_PENT = 5 + 0 shl 4 CPU_SUBTYPE_PENTPRO = 6 + 1 shl 4 CPU_SUBTYPE_PENTII_M3 = 6 + 3 shl 4 CPU_SUBTYPE_PENTII_M5 = 6 + 5 shl 4 CPU_SUBTYPE_PENTIUM_4 = 10 + 0 shl 4 MH_OBJECT = 0x1 MH_EXECUTE = 0x2 MH_FVMLIB = 0x3 MH_CORE = 0x4 MH_PRELOAD = 0x5 MH_DYLIB = 0x6 MH_DYLINKER = 0x7 MH_BUNDLE = 0x8 MH_DYLIB_STUB = 0x9 MH_DSYM = 0xA MH_KEXT_BUNDLE = 0xB MH_NOUNDEFS = 0x1 MH_INCRLINK = 0x2 MH_DYLDLINK = 0x4 MH_BINDATLOAD = 0x8 MH_PREBOUND = 0x10 MH_SPLIT_SEGS = 0x20 MH_LAZY_INIT = 0x40 MH_TWOLEVEL = 0x80 MH_FORCE_FLAT = 0x100 MH_NOMULTIDEFS = 0x200 MH_NOFIXPREBINDING = 0x400 MH_PREBINDABLE = 0x800 MH_ALLMODSBOUND = 0x1000 MH_SUBSECTIONS_VIA_SYMBOLS = 0x2000 MH_CANONICAL = 0x4000 MH_WEAK_DEFINES = 0x8000 MH_BINDS_TO_WEAK = 0x10000 MH_ALLOW_STACK_EXECUTION = 0x20000 MH_ROOT_SAFE = 0x40000 MH_SETUID_SAFE = 0x80000 MH_NO_REEXPORTED_DYLIBS = 0x100000 MH_PIE = 0x200000 MH_DEAD_STRIPPABLE_DYLIB = 0x400000 MH_HAS_TLV_DESCRIPTORS = 0x800000 MH_NO_HEAP_EXECUTION = 0x1000000 MH_APP_EXTENSION_SAFE = 0x2000000 LC_REQ_DYLD = 0x80000000 LC_SEGMENT = 0x1 LC_SYMTAB = 0x2 LC_SYMSEG = 0x3 LC_THREAD = 0x4 LC_UNIXTHREAD = 0x5 LC_LOADFVMLIB = 0x6 LC_IDFVMLIB = 0x7 LC_IDENT = 0x8 LC_FVMFILE = 0x9 LC_PREPAGE = 0xa LC_DYSYMTAB = 0xb LC_LOAD_DYLIB = 0xc LC_ID_DYLIB = 0xd LC_LOAD_DYLINKER = 0xe LC_ID_DYLINKER = 0xf LC_PREBOUND_DYLIB = 0x10 LC_ROUTINES = 0x11 LC_SUB_FRAMEWORK = 0x12 LC_SUB_UMBRELLA = 0x13 LC_SUB_CLIENT = 0x14 LC_SUB_LIBRARY = 0x15 LC_TWOLEVEL_HINTS = 0x16 LC_PREBIND_CKSUM = 0x17 LC_LOAD_WEAK_DYLIB = 0x18 LC_SEGMENT_64 = 0x19 LC_ROUTINES_64 = 0x1a LC_UUID = 0x1b LC_RPATH = 0x1c + LC_REQ_DYLD LC_CODE_SIGNATURE = 0x1d LC_SEGMENT_SPLIT_INFO = 0x1e LC_REEXPORT_DYLIB = 0x1f + LC_REQ_DYLD LC_LAZY_LOAD_DYLIB = 0x20 LC_ENCRYPTION_INFO = 0x21 LC_DYLD_INFO = 0x22 LC_DYLD_INFO_ONLY = 0x22 + LC_REQ_DYLD SG_HIGHVM = 0x1 SG_FVMLIB = 0x2 SG_NORELOC = 0x4 SECTION_TYPE = 0x000000ff SECTION_ATTRIBUTES = 0xffffff00 S_REGULAR = 0x0 S_ZEROFILL = 0x1 S_CSTRING_LITERALS = 0x2 S_4BYTE_LITERALS = 0x3 S_8BYTE_LITERALS = 0x4 S_LITERAL_POINTERS = 0x5 S_NON_LAZY_SYMBOL_POINTERS = 0x6 S_LAZY_SYMBOL_POINTERS = 0x7 S_SYMBOL_STUBS = 0x8 S_MOD_INIT_FUNC_POINTERS = 0x9 S_MOD_TERM_FUNC_POINTERS = 0x0a S_COALESCED = 0x0b S_GB_ZEROFILL = 0x0c S_INTERPOSING = 0x0d S_16BYTE_LITERALS = 0x0e S_DTRACE_DOF = 0x0f S_LAZY_DYLIB_SYMBOL_POINTERS = 0x10 S_THREAD_LOCAL_REGULAR = 0x11 S_THREAD_LOCAL_ZEROFILL = 0x12 S_THREAD_LOCAL_VARIABLES = 0x13 S_THREAD_LOCAL_VARIABLE_POINTERS = 0x14 S_THREAD_LOCAL_INIT_FUNCTION_POINTERS = 0x15 SECTION_ATTRIBUTES_USR = 0xff000000 S_ATTR_PURE_INSTRUCTIONS = 0x80000000 S_ATTR_NO_TOC = 0x40000000 S_ATTR_STRIP_STATIC_SYMS = 0x20000000 S_ATTR_NO_DEAD_STRIP = 0x10000000 S_ATTR_LIVE_SUPPORT = 0x08000000 S_ATTR_SELF_MODIFYING_CODE = 0x04000000 S_ATTR_DEBUG = 0x02000000 SECTION_ATTRIBUTES_SYS = 0x00ffff00 S_ATTR_SOME_INSTRUCTIONS = 0x00000400 S_ATTR_EXT_RELOC = 0x00000200 S_ATTR_LOC_RELOC = 0x00000100 VM_PROT_NONE = 0x00 VM_PROT_READ = 0x01 VM_PROT_WRITE = 0x02 VM_PROT_EXECUTE = 0x04 VM_PROT_DEFAULT = VM_PROT_READ or VM_PROT_WRITE VM_PROT_ALL = VM_PROT_READ or VM_PROT_WRITE or VM_PROT_EXECUTE VM_PROT_NO_CHANGE = 0x08 VM_PROT_COPY = 0x10 VM_PROT_WANTS_COPY = 0x10 x86_THREAD_STATE32 = 1 x86_FLOAT_STATE32 = 2 x86_EXCEPTION_STATE32 = 3 x86_THREAD_STATE64 = 4 x86_FLOAT_STATE64 = 5 x86_EXCEPTION_STATE64 = 6 x86_THREAD_STATE = 7 x86_FLOAT_STATE = 8 x86_EXCEPTION_STATE = 9 x86_DEBUG_STATE32 = 10 x86_DEBUG_STATE64 = 11 x86_DEBUG_STATE = 12 THREAD_STATE_NONE = 13 N_STAB = 0xe0 N_PEXT = 0x10 N_TYPE = 0x0e N_EXT = 0x01 N_UNDF = 0x0 N_ABS = 0x2 N_SECT = 0xe N_PBUD = 0xc N_INDR = 0xa NO_SECT = 0 MAX_SECT = 255 REFERENCE_TYPE = 0xf REFERENCE_FLAG_UNDEFINED_NON_LAZY = 0 REFERENCE_FLAG_UNDEFINED_LAZY = 1 REFERENCE_FLAG_DEFINED = 2 REFERENCE_FLAG_PRIVATE_DEFINED = 3 REFERENCE_FLAG_PRIVATE_UNDEFINED_NON_LAZY = 4 REFERENCE_FLAG_PRIVATE_UNDEFINED_LAZY = 5 REFERENCED_DYNAMICALLY = 0x0010
Then the basic macros for the creation of Mach-O headers:
Code:
; Basic layer: command headers MachO:: namespace MachO if defined Settings.ProcessorType CPUTYPE := Settings.ProcessorType else CPUTYPE := CPU_TYPE_I386 end if if defined Settings.ProcessorSubtype CPUSUBTYPE := Settings.ProcessorSubtype else if CPUTYPE and CPU_ARCH_ABI64 CPUSUBTYPE := CPU_SUBTYPE_I386_ALL + CPU_SUBTYPE_LIB64 else CPUSUBTYPE := CPU_SUBTYPE_I386_ALL end if end if if defined Settings.FileType FILETYPE := Settings.FileType else FILETYPE := MH_EXECUTE end if if defined Settings.Flags FLAGS := Settings.Flags else FLAGS := MH_NOUNDEFS + MH_DYLDLINK end if if defined Settings.SegmentAlignment SEGMENT_ALIGNMENT := Settings.SegmentAlignment else SEGMENT_ALIGNMENT := 1000h end if if defined Settings.FileAlignment FILE_ALIGNMENT := Settings.FileAlignment else FILE_ALIGNMENT := 1000h end if if CPUTYPE and CPU_ARCH_ABI64 magic dd 0xFEEDFACF else magic dd 0xFEEDFACE end if cputype dd CPUTYPE cpusubtype dd CPUSUBTYPE filetype dd FILETYPE ncmds dd NUMBER_OF_COMMANDS sizeofcmds dd SIZE_OF_COMMANDS flags dd FLAGS if CPUTYPE and CPU_ARCH_ABI64 reserved dd ? end if COMMAND_NUMBER = 0 COMMAND_POSITION = $ commands db SIZE_OF_COMMANDS dup 0 org $ macro command type if MachO.COMMAND_POSITION > 0 virtual at MachO.COMMAND_POSITION MachO.COMMAND_POSITION = 0 end if match t, type if MachO.COMMAND_NUMBER > 0 repeat 1, p:MachO.COMMAND_NUMBER MachO.load#p.SIZE := $ - MachO.load#p.cmd end repeat end if MachO.COMMAND_NUMBER = MachO.COMMAND_NUMBER + 1 MachO.COMMAND = $ repeat 1, n:MachO.COMMAND_NUMBER namespace MachO.load#n cmd dd t cmdsize dd SIZE end namespace end repeat end match end macro macro text if MachO.COMMAND_POSITION = 0 MachO.COMMAND_POSITION = $ load MachO.COMMAND_DATA:$-$$ from $$ store MachO.COMMAND_DATA:$-$$ at MachO:$$ end virtual end if end macro end namespace macro align? boundary,value:? db (boundary-1)-($+boundary-1) mod boundary dup value end macro postpone MachO.text namespace MachO if COMMAND_NUMBER > 0 repeat 1, p:COMMAND_NUMBER load#p.SIZE := COMMAND_POSITION - MachO.load#p.cmd end repeat end if NUMBER_OF_COMMANDS := COMMAND_NUMBER SIZE_OF_COMMANDS := COMMAND_POSITION - commands end namespace end postpone
The MachO.command and MachO.text macros allow to switch between defining structures in the Mach-O headers and the actual text of program. The MachO.command can have an argument specifying the type of load command to create (it should be one of the LC_* constants), it then automatically creates and fills the "cmd" and "cmdsize" fields for that command. This little automation allows to manually create various structures needed for a functional Mach-O executable in a quite manageable way. Here's an example:
Code:
include '80386.inc' use32 include 'macho.inc' MachO.command LC_SEGMENT segname ddq '__TEXT' vmaddr dd 0 vmsize dd SEGMENT_SIZE fileoff dd 0 filesize dd SEGMENT_SIZE maxprot dd VM_PROT_ALL initprot dd VM_PROT_READ or VM_PROT_EXECUTE nsects dd 2 flags dd 0 MachO.command section1: .sectname ddq '__text' .segname ddq '__TEXT' .addr dd SECTION1_ADDRESS .size dd SECTION1_SIZE .offset dd SECTION1_OFFSET .align dd 4 .reloff dd 0 .nreloc dd 0 .flags dd 0 .reserved1 dd ? .reserved2 dd ? MachO.text align 16 SECTION1_ADDRESS := $ SECTION1_OFFSET := $% SYS_exit = 1 SYS_write = 4 start: push msg.length push msg push 1 mov eax,SYS_write sub esp,4 int 0x80 add esp,4*4 push 0 mov eax,SYS_exit sub esp,4 int 0x80 SECTION1_SIZE := $ - SECTION1_ADDRESS MachO.command section2: .sectname ddq '__cstring' .segname ddq '__TEXT' .addr dd SECTION2_ADDRESS .size dd SECTION2_SIZE .offset dd SECTION2_OFFSET .align dd 2 .reloff dd 0 .nreloc dd 0 .flags dd 0 .reserved1 dd ? .reserved2 dd ? MachO.text align 4 SECTION2_ADDRESS := $ SECTION2_OFFSET := $% msg db 'Hello World!',0Ah .length = $ - msg db 1000h-$ dup 0 ; pad file to a minimum of 4096 bytes required by modern MacOS versions SECTION2_SIZE := $ - SECTION2_ADDRESS SEGMENT_SIZE := $ MachO.command LC_UNIXTHREAD thread_state: .flavor dd 1 ; i386_THREAD_STATE .count dd 16 ; i386_THREAD_STATE_COUNT .eax dd 0 .ebx dd 0 .ecx dd 0 .edx dd 0 .edi dd 0 .esi dd 0 .ebp dd 0 .esp dd 0 .ss dd 0 .eflags dd 0 .eip dd start .cs dd 0 .ds dd 0 .es dd 0 .fs dd 0 .gs dd 0
The next layer of macro automates the creation of segments, sections and thread initialization structures:
Code:
; Intermediate layer: segments and sections namespace MachO if defined Settings.BaseAddress VM_ADDRESS = Settings.BaseAddress if CPUTYPE and CPU_ARCH_ABI64 MachO.command LC_SEGMENT_64 else MachO.command LC_SEGMENT end if namespace MachO.segment1 segname emit 16: '__PAGEZERO' if CPUTYPE and CPU_ARCH_ABI64 vmaddr dq 0 vmsize dq VM_ADDRESS fileoff dq 0 filesize dq 0 else vmaddr dd 0 vmsize dd VM_ADDRESS fileoff dd 0 filesize dd 0 end if maxprot dd 0 initprot dd 0 nsects dd 0 flags dd 0 end namespace SEGMENT_NUMBER = 1 else VM_ADDRESS = 0 SEGMENT_NUMBER = 0 end if TEXT_OFFSET = $% end namespace macro MachO.segment declaration MachO.close_segment local NAME,VMADDR,VMSIZE,FILEOFF,FILESIZE,MAXPROT,INITPROT,NSECTS INITPROT = VM_PROT_NONE MAXPROT = VM_PROT_ALL match name attributes, declaration NAME = name local sequence define sequence attributes: while 1 match attribute tail, sequence match =readable?, attribute INITPROT = INITPROT or VM_PROT_READ MAXPROT = MAXPROT or VM_PROT_READ else match =notreadable?, attribute INITPROT = INITPROT and not VM_PROT_READ MAXPROT = MAXPROT and not VM_PROT_READ else match =writeable?, attribute INITPROT = INITPROT or VM_PROT_WRITE MAXPROT = MAXPROT or VM_PROT_WRITE else match =writable?, attribute INITPROT = INITPROT or VM_PROT_WRITE MAXPROT = MAXPROT or VM_PROT_WRITE else match =notwriteable?, attribute INITPROT = INITPROT and not VM_PROT_WRITE MAXPROT = MAXPROT and not VM_PROT_WRITE else match =notwritable?, attribute INITPROT = INITPROT and not VM_PROT_WRITE MAXPROT = MAXPROT and not VM_PROT_WRITE else match =executable?, attribute INITPROT = INITPROT or VM_PROT_EXECUTE MAXPROT = MAXPROT or VM_PROT_EXECUTE else match =notexecutable?, attribute INITPROT = INITPROT and not VM_PROT_EXECUTE MAXPROT = MAXPROT and not VM_PROT_EXECUTE else err 'invalid argument' end match redefine sequence tail else break end match end while else NAME = definition end match if lengthof NAME > 16 err 'name to long' end if VMADDR := MachO.VM_ADDRESS if $% = MachO.TEXT_OFFSET & FILESIZE > 0 MachO.FILE_OFFSET = 0 else MachO.FILE_OFFSET = $% end if if FILESIZE FILEOFF := MachO.FILE_OFFSET else FILEOFF := 0 end if MachO.SEGMENT_NUMBER = MachO.SEGMENT_NUMBER + 1 if MachO.CPUTYPE and CPU_ARCH_ABI64 MachO.command LC_SEGMENT_64 else MachO.command LC_SEGMENT end if repeat 1, s:MachO.SEGMENT_NUMBER namespace MachO.segment#s segname emit 16: NAME if MachO.CPUTYPE and CPU_ARCH_ABI64 vmaddr dq VMADDR vmsize dq VMSIZE fileoff dq FILEOFF filesize dq FILESIZE else vmaddr dd VMADDR vmsize dd VMSIZE fileoff dd FILEOFF filesize dd FILESIZE end if maxprot dd MAXPROT initprot dd INITPROT nsects dd NSECTS flags dd 0 end namespace repeat NSECTS namespace MachO.segment#s#_section#% sectname emit 16: ? segname emit 16: ? if MachO.CPUTYPE and CPU_ARCH_ABI64 addr dq ? size dq ? else addr dd ? size dd ? end if offset dd ? align. dd ? reloff dd ? nreloc dd ? flags dd ? reserved1 dd ? reserved2 dd ? if MachO.CPUTYPE and CPU_ARCH_ABI64 dd ? end if end namespace end repeat end repeat macro MachO.close_segment MachO.close_section local OFFSET,NEXT,TOP,FILL TOP = $% NSECTS := MachO.SECTION_NUMBER if MachO.FILE_OFFSET >= $%% FILL = MachO.FILE_OFFSET else FILL = $%% end if repeat 1, s:MachO.SEGMENT_NUMBER repeat NSECTS, n:2 if % = 1 load OFFSET from MachO:MachO.segment#s#_section#%.offset else OFFSET = NEXT end if if % < %% load NEXT from MachO:MachO.segment#s#_section#n.offset else NEXT = TOP end if if OFFSET >= $%% store S_ZEROFILL at MachO:MachO.segment#s#_section#%.flags store 0 at MachO:MachO.segment#s#_section#%.size else FILL = NEXT end if end repeat end repeat FILESIZE := FILL - MachO.FILE_OFFSET FILL = FILL - $%% MachO.SECTION_BYPASS = 1 section MachO.VM_ADDRESS + FILESIZE - FILL restore MachO.SECTION_BYPASS db FILL dup 0 if FILESIZE rb (MachO.FILE_ALIGNMENT-$%) and (MachO.FILE_ALIGNMENT-1) end if MachO.VM_ADDRESS = MachO.VM_ADDRESS + TOP - MachO.FILE_OFFSET MachO.VM_ADDRESS = MachO.VM_ADDRESS + (MachO.SEGMENT_ALIGNMENT-MachO.VM_ADDRESS) and (MachO.SEGMENT_ALIGNMENT-1) VMSIZE := MachO.VM_ADDRESS - VMADDR purge MachO.close_segment end macro MachO.text org VMADDR + $% - MachO.FILE_OFFSET MachO.SECTION_NUMBER = 0 end macro macro MachO.section declaration if MachO.SEGMENT_NUMBER = 0 MachO.segment '__TEXT' readable writable executable end if MachO.close_section local SECTNAME,SEGNAME,FLAGS,ALIGN,RESERVED1,RESERVED2 FLAGS = S_REGULAR ALIGN = 0 RESERVED1 = 0 RESERVED2 = 0 local sequence match segname:sectname tail, declaration: SECTNAME = sectname SEGNAME = segname define sequence tail else match name tail, declaration: SECTNAME = name repeat 1, s:MachO.SEGMENT_NUMBER load SEGNAME from MachO:MachO.segment#s.segname end repeat define sequence tail end match while 1 match :, sequence break else match =align? boundary tail, sequence ALIGN = bsf boundary if bsf boundary <> bsr boundary err 'invalid alignment value' end if redefine sequence tail else match =flags?(value) tail, sequence FLAGS = value redefine sequence tail else match =reserved1?(value) tail, sequence RESERVED1 = value redefine sequence tail else match =reserved2?(value) tail, sequence RESERVED2 = value redefine sequence tail else err 'invalid arguments' end match end while MachO.text if ALIGN align 1 shl ALIGN end if MachO.SECTION_NUMBER = MachO.SECTION_NUMBER + 1 repeat 1, s:MachO.SEGMENT_NUMBER, t:MachO.SECTION_NUMBER if ~ defined MachO.segment#s#_section#t.sectname MachO.command namespace MachO.segment#s#_section#% sectname emit 16: ? segname emit 16: ? if MachO.CPUTYPE and CPU_ARCH_ABI64 addr dq ? size dq ? else addr dd ? size dd ? end if offset dd ? align. dd ? reloff dd ? nreloc dd ? flags dd ? reserved1 dd ? reserved2 dd ? if MachO.CPUTYPE and CPU_ARCH_ABI64 dd ? end if end namespace MachO.text end if store SECTNAME at MachO:MachO.segment#s#_section#t.sectname store SEGNAME at MachO:MachO.segment#s#_section#t.segname store $ at MachO:MachO.segment#s#_section#t.addr store $% at MachO:MachO.segment#s#_section#t.offset store FLAGS at MachO:MachO.segment#s#_section#t.flags store ALIGN at MachO:MachO.segment#s#_section#t.align store RESERVED1 at MachO:MachO.segment#s#_section#t.reserved1 store RESERVED2 at MachO:MachO.segment#s#_section#t.reserved2 end repeat end macro macro MachO.close_section MachO.text if MachO.SECTION_NUMBER local SIZE repeat 1, s:MachO.SEGMENT_NUMBER, t:MachO.SECTION_NUMBER load OFFSET from MachO:MachO.segment#s#_section#t.offset store $%-OFFSET at MachO:MachO.segment#s#_section#t.size end repeat end if end macro macro MachO.close_segment end macro postpone MachO.close_segment if $%% < 1000h store 0:byte at $-1 ; enforce minimum file size for OS X 10.10.5 and higher end if end postpone macro segment? args& MachO.segment args end macro macro section? args& if defined MachO.SECTION_BYPASS section args else MachO.section args end if end macro macro entry? regs& iterate reg, regs match name:value, reg MachO.thread.name? := value else if MachO.CPUTYPE = CPU_TYPE_I386 MachO.thread.eip? := reg else if MachO.CPUTYPE = CPU_TYPE_X86_64 MachO.thread.rip? := reg end if end iterate MachO.command LC_UNIXTHREAD if MachO.CPUTYPE = CPU_TYPE_I386 MachO.thread.flavor dd x86_THREAD_STATE32 iterate name, eax,ebx,ecx,edx,edi,esi,ebp,esp,ss,eflags,eip,cs,ds,es,fs,gs if % = 1 MachO.thread.count dd %% end if if defined MachO.thread.name? dd MachO.thread.name? else dd ? end if end iterate else if MachO.CPUTYPE = CPU_TYPE_X86_64 MachO.thread.flavor dd x86_THREAD_STATE64 iterate name, rax,rbx,rcx,rdx,rdi,rsi,rbp,rsp,r8,r9,r10,r11,r12,r13,r14,r15,rip,rflags,cs,fs,gs if % = 1 MachO.thread.count dd %%*2 end if if defined MachO.thread.name? dq MachO.thread.name? else dq ? end if end iterate else err 'CPU not supported' end if MachO.text end macro
With these additional macros the earlier example can be greatly simplified:
Code:
include '80386.inc' use32 include 'macho.inc' segment '__TEXT' readable executable section '__text' SYS_exit = 1 SYS_write = 4 entry $ push msg.length push msg push 1 mov eax,SYS_write sub esp,4 int 0x80 add esp,4*4 push 0 mov eax,SYS_exit sub esp,4 int 0x80 section '__cstring' msg db 'Hello World!',0Ah .length = $ - msg
Note that the sections are subdivisions of segments, and the segments define continuous areas in program memory that have specified access permissions. This is analogous to the corresponding structures in ELF format, where segments and sections can co-exist in a similar way, though fasm's formatters never allowed to create both in the same file - because ELF segments are optional (and mostly useless) in object files, and ELF sections are optional in executable file. In case of Mach-O it is not possible to create section without a parent segment. But it is allowed to create segment with no sections - in the above example all the "section" lines could be removed and it would still work.

In addition to traditional fasm's "readable", "writeable" and "executable" permission keywords there are also "notreadable", "notwriteable", and "notexecutable". This is because Mach-O segments have separate fields for initial and maximum allowable permissions. The use of negated keywords causes permissions to be removed even from the "maxprot" field.

There is a convention to create the "__PAGEZERO" segment at address 0 with all permissions disabled, mainly as a protection against accidental use of null pointers. It is possible to prepare such segment manually, like:
Code:
include '80386.inc' use32 include 'macho.inc' segment '__PAGEZERO' notreadable notwritable notexecutable rb 1000h segment '__TEXT' readable executable entry $ push msg.length push msg push 1 mov eax,4 sub esp,4 int 0x80 add esp,4*4 push 0 mov eax,1 sub esp,4 int 0x80 msg db 'Hello World!',0Ah .length = $ - msg
But it is also possible to have it made automatically, by specifying the "BaseAddress" setting for executable. The macros then create "_PAGEZERO" segment spanning the entire area from 0 up to the given base address:
Code:
include '80386.inc' use32 MachO.Settings.BaseAddress = 0x1000 include 'macho.inc' segment '__TEXT' readable executable entry $ push msg.length push msg push 1 mov eax,4 sub esp,4 int 0x80 add esp,4*4 push 0 mov eax,1 sub esp,4 int 0x80 msg db 'Hello World!',0Ah .length = $ - msg
And by altering the "ProcessorType" setting it is possible to create 64-bit executables:
Code:
include 'x64.inc' use64 MachO.Settings.ProcessorType equ CPU_TYPE_X86_64 MachO.Settings.BaseAddress = 0x100000000 include 'macho.inc' segment '__TEXT' readable executable SYSCALL_CLASS_UNIX = 2 SYSCALL_CLASS_SHIFT = 24 define SYSCALL_CONSTRUCT_UNIX SYSCALL_CLASS_UNIX shl SYSCALL_CLASS_SHIFT + SYS_exit = 1 SYS_write = 4 entry $, rdi: 1, rsi: msg mov rdx,msg.length mov rax,SYSCALL_CONSTRUCT_UNIX(SYS_write) syscall mov rdi,rax mov rax,SYSCALL_CONSTRUCT_UNIX(SYS_exit) syscall msg db 'This is a 64-bit test!',0Ah,0 .length = $ - msg
This 64-bit example uses the Unix syscall convention for a change, and it also utilizes the LC_UNIXTHREAD command created by the "entry" macro to set up a few more registers beside RIP. For some reason this is not a reliable method in general, it does not work for some of the registers, but for RSI and RDI it seems to be working.


Last edited by Tomasz Grysztar on 11 Aug 2017, 16:26; edited 3 times in total
Post 10 Aug 2017, 18:27
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6862
Location: Kraków, Poland
The further layer of macros is for the dynamic linking. This is a dangerous ground, the structures required for this are numerous and cross-reference each other, and on the top of that, recent versions of MacOS introduced an entirely new load commands (LC_DYLD) for this purpose. I have not yet touched these new commands (which appear to contain some kind of byte code interpreted by dylink), for now I tried to get the legacy ones to work.

First I tried the manual creation of symbol tables and dynamic linking structures:
Code:
include '80386.inc' use32 MachO.Settings.BaseAddress = 0x1000 include 'macho.inc' segment '__TEXT' readable executable section '__text' align 16 entry start start: and esp,0FFFFFFF0h sub esp,10h mov dword [esp],msg call printf and dword [esp],0 call exit section '__cstring' align 4 msg db 'Hello World!',0Ah,0 segment '__IMPORT' readable executable section '__nl_symbol_ptr' flags(S_NON_LAZY_SYMBOL_POINTERS) reserved1(0) __printf dd ? __exit dd ? section '__jump_table' flags(S_SYMBOL_STUBS+S_ATTR_SOME_INSTRUCTIONS+S_ATTR_PURE_INSTRUCTIONS) reserved1(0) reserved2(STUB_SIZE) printf: jmp dword [__printf] STUB_SIZE := $ - printf exit: jmp dword [__exit] segment '__LINKEDIT' readable symbols = $% ; msg dd str_msg ; n_strx db N_SECT ; n_type db 2 ; n_sect dw 0 ; n_desc dd msg ; n_value ; start dd str_start db N_SECT + N_EXT db 1 dw 0 dd start ; _printf dd str_printf db N_EXT db 0 dw REFERENCE_FLAG_UNDEFINED_NON_LAZY dd 0 ; _exit dd str_exit db N_EXT db 0 dw REFERENCE_FLAG_UNDEFINED_NON_LAZY dd 0 indirect = $% ; indexes into symbol table: dd 2 ; _printf dd 3 ; _exit strings = $% db ' ',0 str_msg = $% - strings db 'msg',0 str_start = $% - strings db 'start',0 str_printf = $% - strings db '_printf',0 str_exit = $% - strings db '_exit',0 strings_size = $% - strings MachO.command LC_LOAD_DYLINKER dd dylinker-MachO.COMMAND dylinker db '/usr/lib/dyld',0 align 4 MachO.command LC_LOAD_DYLIB dd dylib-MachO.COMMAND dd 2 ; timestamp dd 0x006F0103 ; currentver dd 0x00010000 ; compatver dylib db '/usr/lib/libSystem.B.dylib',0 align 4 MachO.command LC_SYMTAB dd symbols ; symoff dd 4 ; nsyms dd strings ; stroff dd strings_size ; strsize MachO.command LC_DYSYMTAB ; one local symbol (msg): dd 0 ; ilocalsym dd 1 ; nlocalsym ; one public symbol (start): dd 1 ; iextdefsym dd 1 ; nextdefsym ; two external symbols (printf, exit): dd 2 ; iundefsym dd 2 ; nundefsym dd 0 ; tocoff dd 0 ; ntoc dd 0 ; modtaboff dd 0 ; nmodtab dd 0 ; extrefsymoff dd 0 ; nextrefsyms ; all external symbols are linked indirectly: dd indirect ; indirectsymoff dd 2 ; nindirectsyms dd 0 ; extreloff dd 0 ; nextrel dd 0 ; locreloff dd 0 ; nlocrel
This seems to work, though I'm still curious what should it look like to use lazy symbol pointers. Apparently this would be a more usual method of linking external functions, but it requires some additional quirks that I are out of my grasp at the moment. And this is still just using the legacy load commands!

The macros I made for dynamic linking use the non-lazy pointers like the above sample:
Code:
; Upper layer: dynamic linking namespace MachO NSYMS = 0 LIB_NUMBER = 0 end namespace macro interpreter? path MachO.command LC_LOAD_DYLINKER namespace MachO.dylinker_command lc_str dd dylinker-MachO.COMMAND dylinker db path,0 align 4 end namespace end macro macro uses? lib& MachO.LIB_NUMBER = MachO.LIB_NUMBER + 1 MachO.command LC_LOAD_DYLIB repeat 1, l:MachO.LIB_NUMBER namespace MachO.dylib#l#_command lc_str dd dylib-MachO.COMMAND timestamp dd 2 match path (a.b.c=,x.y.z), lib current_version dd (x and 0FFFFh) shl 16 + y shl 8 + z compatibility_version dd (a and 0FFFFh) shl 16 + b shl 8 + c dylib db path,0 else current_version dd 10000h compatibility_version dd 10000h dylib db lib,0 end match align 4 end namespace end repeat end macro macro import? definitions& iterate <name,string>, definitions MachO.NSYMS = MachO.NSYMS + 1 define MachO.symbol name name.extrn := 1 name.type := N_EXT name.desc := REFERENCE_FLAG_UNDEFINED_NON_LAZY define name.str string end iterate end macro postpone if MachO.NSYMS segment '__IMPORT' readable writable executable section '__nl_symbol_ptr' flags(S_NON_LAZY_SYMBOL_POINTERS) reserved1(0) align 4 irpv sym, MachO.symbol if sym.extrn if MachO.CPUTYPE and CPU_ARCH_ABI64 sym.ptr dq ? else sym.ptr dd ? end if end if end irpv section '__jump_table' flags(S_SYMBOL_STUBS+S_ATTR_SOME_INSTRUCTIONS+S_ATTR_PURE_INSTRUCTIONS) reserved1(0) reserved2(MachO.JUMP_SIZE) align 16 irpv sym, MachO.symbol if sym.extrn sym: jmp [sym.ptr] if % = 1 MachO.JUMP_SIZE := $ - sym end if end if end irpv segment '__LINKEDIT' readable MachO.SYMOFF := $% irpv sym, MachO.symbol namespace MachO.nlist#% n_strx dd sym.strx n_type db sym.type n_sect db 0 n_desc dw sym.desc if MachO.CPUTYPE and CPU_ARCH_ABI64 n_value dq sym else n_value dd sym end if end namespace end irpv MachO.INDIRECTSYMOFF := $% irpv sym, MachO.symbol if sym.extrn dd %-1 end if end irpv MachO.STROFF := $% db 20h,0 irpv sym, MachO.symbol sym.strx := $% - MachO.STROFF db string sym.str, 0 end irpv MachO.STRSIZE := $% - MachO.STROFF MachO.command LC_SYMTAB namespace MachO.symtab_command symoff dd SYMOFF nsyms dd NSYMS stroff dd STROFF strsize dd STRSIZE end namespace MachO.command LC_DYSYMTAB namespace MachO.dysymtab_command ilocalsym dd 0 nlocalsym dd 0 iextdefsym dd 0 nextdefsym dd 0 iundefsym dd 0 nundefsym dd NSYMS tocoff dd 0 ntoc dd 0 modtaboff dd 0 nmodtab dd 0 extrefsymoff dd 0 nextrefsyms dd 0 indirectsymoff dd INDIRECTSYMOFF nindirectsyms dd NSYMS extreloff dd 0 nextrel dd 0 locreloff dd 0 nlocrel dd 0 end namespace end if end postpone
The syntax enabled by these macros is similar to the ones I made for dynamicallly linked ELF executables:
Code:
include '80386.inc' use32 MachO.Settings.BaseAddress = 0x1000 include 'macho.inc' interpreter '/usr/lib/dyld' uses '/usr/lib/libSystem.B.dylib' (1.0.0, 1225.0.0) import printf,'_printf',\ exit,'_exit' segment '__TEXT' readable executable section '__text' align 16 entry start start: and esp,0FFFFFFF0h sub esp,10h mov dword [esp],msg call printf and dword [esp],0 call exit section '__cstring' align 4 msg db 'Hello World!',0Ah,0
What was not present in ELF is the inclusion of two version numbers for every linked library. The first one is "compatibility version" and the second one is "current version". You can see their values in an executable with "otool -L" command.
Post 10 Aug 2017, 18:48
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6862
Location: Kraków, Poland
I'm attaching the complete set of macros I have made so far, packaged with a few examples.

I was not able to get working 64-bit executable with dynamic linking, it needs an additional investigation. If you find any way to correct or improve these macros, please let me know.


Description: Mach-O executable formatting macros for fasmg, with examples
Download
Filename: machO.zip
Filesize: 8.88 KB
Downloaded: 71 Time(s)

Post 10 Aug 2017, 19:06
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6862
Location: Kraków, Poland
Finally, the above macros can be used to assemble fasmg itself as a dynamically linked Mach-O executable. I'm attaching a pre-assembled executable and the source files that need to be added to the standard "source" directory of fasmg.

EDIT: this is now included in the official fasmg packaging.


Last edited by Tomasz Grysztar on 26 Aug 2017, 15:42; edited 2 times in total
Post 10 Aug 2017, 21:07
View user's profile Send private message Visit poster's website Reply with quote
ProMiNick



Joined: 24 Mar 2012
Posts: 157
Location: Russian Federation, Sochi
fasmg sources not full:
Code:
flat assembler version g.hvj4j C:\FASM\SOURCE_G\fasmg_macos\fasmg.asm [66]: or al,TRACE_ERROR_STACK macro or [2] macro parse_operand [10] macro parse_operand_value [31]: ns.imm = +op Processed: @src.imm = +TRACE_ERROR_STACK Error: symbol 'TRACE_ERROR_STACK' is undefined or out of scope.


pushes & popes in machO - cancelled? Why so strange ccall?
Post 10 Aug 2017, 21:46
View user's profile Send private message Send e-mail Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6862
Location: Kraków, Poland
ProMiNick wrote:
fasmg sources not full:
Code:
flat assembler version g.hvj4j C:\FASM\SOURCE_G\fasmg_macos\fasmg.asm [66]: or al,TRACE_ERROR_STACK macro or [2] macro parse_operand [10] macro parse_operand_value [31]: ns.imm = +op Processed: @src.imm = +TRACE_ERROR_STACK Error: symbol 'TRACE_ERROR_STACK' is undefined or out of scope.
Please make sure you use the latest fasmg source base. It assembles correctly with version hvvox.
Post 10 Aug 2017, 21:57
View user's profile Send private message Visit poster's website Reply with quote
ProMiNick



Joined: 24 Mar 2012
Posts: 157
Location: Russian Federation, Sochi
Yes. All compiled OK.


What about ccall? pushes? why moves to [esp...] used?
Post 10 Aug 2017, 22:24
View user's profile Send private message Send e-mail Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6862
Location: Kraków, Poland
ProMiNick wrote:
What about ccall? pushes? why moves to [esp...] used?
This is a variant of "ccall" intended only for fasmg sources, because SYSTEM.INC always uses it with simple (register/constant) arguments and memory operands are not supported. The setting up of stack frame is similar to the one of fastcall convention, I have done it this way because it allows to allocate stack space and align it at the same time. The alignment is necessary for MacOS, previous patches also had to provide it in some way.

PS I have updated the macros above with a few small fixes.
Post 11 Aug 2017, 09:47
View user's profile Send private message Visit poster's website Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 714
Oh thank you Tomasz! I just went through the trouble of using objconv.
Some questions:
If I port the code reasonably from fasm to fasmg, will fasmg produce identical binaries? Does fasmg do all of the size optimisations that fasm does?
Post 11 Aug 2017, 15:32
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6862
Location: Kraków, Poland
tthsqe wrote:
If I port the code reasonably from fasm to fasmg, will fasmg produce identical binaries? Does fasmg do all of the size optimisations that fasm does?
The x86 macros that come with the base package reproduce the encoding and optimizations of fasm 1 quite faithfully. For example, self-assembled fasmg is identical to the binary assembled with fasm 1 (the only differences are in the PE headers, where there are things like timestamp).

Of course it is also possible that there may be some alternative macro sets implementing x86 in the future. I think someone on the board worked on one already. A less complex one could be made to allow faster assembly (though there may be not much to gain, implementation of instructions as macros is going to be relatively slow by its very nature). On the opposite end of spectrum, I started working on even more complex macros that demonstrate some of the ideas I had for fasm 2 (before I decided to reduce my fasm 2 project to just fasmg).
Post 11 Aug 2017, 16:13
View user's profile Send private message Visit poster's website Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 714
I will give porting my source from fasm to fasmg a shot then. BTW, I know how much trouble it is to get documentation on apple's suff. Any they change things from version to version.
Post 11 Aug 2017, 16:32
View user's profile Send private message Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 714
I noticed that your mach-o macros don't contain anything resembling relocations. Could I get confirmation/disconfirmation of the following concerning the code at the end?

0. In any format:
The d address in f is encoded as rip-relative since fasm knows the sizes of the sections.

1. In format PE64:
The d address in g cannot be encoded as rip-relative by an architecture constraint. In the dword field of the encoding, fasm puts the absolute address of d since the absolute address of the base of the module is known. If the absolute address of d cannot be encoded as a dword, an error occurs.

2. In format PE64 with fixups:
The d address in g is encoded using an assumed absolute address (p) of the base of the module. If the OS decides to load the module at another address (q), then "data fixups; end data" contains the locations of the dwords/qwords that need to be shifted by q-p. If any of these operations overflows, an error occurs.

3. In format ELF64:
The d address in g is encoded using "dd ?". When the OS loads the module at some address (p), then the relocation section contains the locations of the dwords/qwords that need to modified based on the value of p. If any of these operations overflows, an error occurs.

4. In format MACH-O:
Much the same as "format PE64"? Of course MACH-O has some relocation scheme so that dynamically linked libraries are possible?

Code:
...[some section]... f: mov eax, dword[d] ret g: xor eax, eax mov eax, dword[d+rax] ret ...[another section]... d: dd 42
Post 12 Aug 2017, 07:38
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6862
Location: Kraków, Poland
tthsqe wrote:
I noticed that your mach-o macros don't contain anything resembling relocations.
I mentioned in the beginning that I started with the executable variant of this format, hoping that it would be quicker and simpler to implement than a linkable object. But, while Mach-O object may be similar to ELF object in implementation, executable turned out much more complex than ELF (as a quick look at the ELFEXE.INC may prove).

tthsqe wrote:
Could I get confirmation/disconfirmation of the following concerning the code at the end?
Yes, but first let me explain the capabilities of different formats in general.

All of the three PE/COFF, ELF, and Mach-O formats have both the executable and object variant (in case of PE/COFF the executable variant is PE and the object variant is COFF).

The executable variant is always an image assembled with a fixed memory layout and an assumed base address. Optionally the image may allow to be loaded at different address, either because it is position-independent or because it has fixups that can be applied to correct the addresses in code when the base shifts. Only PE format has the ability to contain fixups, in case of ELF and Mach-O if the executable is to allow being loaded at different base, in needs to be PIC.

All of the executable formats also allow dynamic linking, which involves resolving some external symbols and placing their addresses in a special data table in the executable image. Most of the time these are the addresses of external functions, and calling these functions can be done either though an indirect call to the field in the dynamic linking table:
Code:
call [__exit]
or through a gate:
Code:
call exit ; ... exit: jmp [_exit]

As for the object formats, they always have true relocations.

tthsqe wrote:
0. In any format:
The d address in f is encoded as rip-relative since fasm knows the sizes of the sections.
First: this is only true for 64-bit code. In 32-bit mode f would use absolute addressing.
Second: even RIP-relative addressing may still require relocation, when the output is a relocatable object and f and d reside in different sections. In the process of linking the sections may end up at different distance from each other and relative addressing still requires relocation. This applies to RIP-relative data addressing, and to the relative jumps/calls as well.

tthsqe wrote:
1. In format PE64:
The d address in g cannot be encoded as rip-relative by an architecture constraint. In the dword field of the encoding, fasm puts the absolute address of d since the absolute address of the base of the module is known. If the absolute address of d cannot be encoded as a dword, an error occurs.
This is true for any executable format with no fixups, so for executable ELF or Mach-O it is the same (as they have no option to contain fixups).

tthsqe wrote:
2. In format PE64 with fixups:
The d address in g is encoded using an assumed absolute address (p) of the base of the module. If the OS decides to load the module at another address (q), then "data fixups; end data" contains the locations of the dwords/qwords that need to be shifted by q-p. If any of these operations overflows, an error occurs.
Yes, and this is unique to PE (I would say it is one of its strong points).

tthsqe wrote:
3. In format ELF64:
The d address in g is encoded using "dd ?". When the OS loads the module at some address (p), then the relocation section contains the locations of the dwords/qwords that need to modified based on the value of p. If any of these operations overflows, an error occurs.
This is what happens in case of object files, and it is the same for kinds of objects - COFF, ELF, or Mach-O.

In general Mach-O format capabilities are similar to ELF - the executable variant allows dynamic linking (this includes things like GOT for PIC) but no fixups, the object variant has true relocations. All the examples I created so far are executables (with MH_EXECUTABLE type in header), but you could use the same low-layer macros to create an object file. You'd need to alter some fields in the header with settings like:
Code:
MachO.Settings.FileType equ MH_OBJECT MachO.Settings.Flags = 0
and then perhaps alter the SEGMENT/SECTION macros a little, to use relocatable virtual address (like ELF64.INC) and generate relocation tables for each section.
Post 12 Aug 2017, 14:09
View user's profile Send private message Visit poster's website Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 714
Thomas, your 64 macros don't work for dynamic linking.

I have attached the binary of a working executable as well as some of its info with objconv. Your macros are missing a few sections as well as the trampoline instructions for the dynamic loader. This program calls write then exit.
In the program, "write" and "exit" are calls to jmp instructions in the ._TEXT.__stubs section. Each of these jumps fetches an address from the ._DATA.__la_symbol_ptr section. Initially, these address in the ._DATA.__la_symbol_ptr point to something like
Code:
push N jmp ?_004

where ?_004 is the address of the thing that calls imp_dyld_stub_binder. If you look in the binary, your will see strings "@_exit" and "@_write" in this order. The "N" is the offset of function name from the first name (@_exit). As for how imp_dyld_stub_binder knows where to write the address of the found function to the ._DATA.__la_symbol_ptr section, I have no idea. (I am fairly certain that imp_dyld_stub_binder will overwrite these addresses so that it is not called the next time the imported function with this name is called).

Also, objconv completed with some error, so the human readable form might not be complete. I have been reading http://timetobleed.com/dynamic-linking-elf-vs-mach-o/

Code:
; Disassembly of file: mac_test ; Mode: 64 bits ; Syntax: YASM/NASM ; Instruction set: 8086, x64 default rel global __mh_execute_header global _main extern dyld_stub_binder ; byte extern _write ; byte extern _exit ; byte SECTION ._TEXT._code align=64 execute ; section number 1, code _main: ; Function begin and rsp, 0FFFFFFFFFFFFFFF0H ; 0F40 _ 48: 83. E4, F0 mov edi, 1 ; 0F44 _ BF, 00000001 lea rsi, [rel ?_001] ; 0F49 _ 48: 8D. 35, 00000011(rel) mov edx, 13 ; 0F50 _ BA, 0000000D call ?_003 ; 0F55 _ E8, 0000001A(rel) mov eax, edi ; 0F5A _ 89. F8 call ?_002 ; 0F5C _ E8, 0000000D(rel) ?_001: ; Error: Prefix after REX prefix not allowed ; Note: Prefix bit or byte has no meaning in this context ; insb ; 0F61 _ 48 65: 6C db 48H, 65H, 6CH ; insb ; 0F64 _ 6C db 6CH ; outsd ; 0F65 _ 6F db 6FH ; and byte [rdi+6FH], dl ; 0F66 _ 20. 57, 6F db 20H, 57H, 6FH ; jc 97H ; 0F69 _ 72, 6C db 72H, 6CH ; Note: Function does not end with ret or jmp ; and dword [fs:rdx], ecx ; 0F6B _ 64: 21. 0A db 64H, 21H, 0AH ; _main End of function SECTION ._TEXT.__stubs align=2 execute ; section number 2, code ?_002: ; Local function jmp near [rel ?_005] ; 0F6E _ FF. 25, 0000009C(rel) ?_003: ; Local function jmp near [rel ?_006] ; 0F74 _ FF. 25, 0000009E(rel) SECTION ._TEXT.__stub_helper align=4 execute ; section number 3, code ?_004: ; Local function lea r11, [rel imp__exit] ; 0F7C _ 4C: 8D. 1D, 00000085(rel) push r11 ; 0F83 _ 41: 53 jmp near [rel imp_dyld_stub_binder] ; 0F85 _ FF. 25, 00000075(rel) nop ; 0F8B _ 90 ; Note: Immediate operand could be made smaller by sign extension push 0 ; 0F8C _ 68, 00000000 ; Note: Immediate operand could be made smaller by sign extension jmp ?_004 ; 0F91 _ E9, FFFFFFE6 ; Note: Immediate operand could be made smaller by sign extension push 12 ; 0F96 _ 68, 0000000C ; Note: Immediate operand could be made smaller by sign extension jmp ?_004 ; 0F9B _ E9, FFFFFFDC SECTION ._TEXT.__unwind_info align=4 noexecute ; section number 4, data db 01H, 00H, 00H, 00H, 1CH, 00H, 00H, 00H ; 0FA0 _ ........ db 00H, 00H, 00H, 00H, 1CH, 00H, 00H, 00H ; 0FA8 _ ........ db 00H, 00H, 00H, 00H, 1CH, 00H, 00H, 00H ; 0FB0 _ ........ db 02H, 00H, 00H, 00H, 40H, 0FH, 00H, 00H ; 0FB8 _ ....@... db 34H, 00H, 00H, 00H, 34H, 00H, 00H, 00H ; 0FC0 _ 4...4... db 6FH, 0FH, 00H, 00H, 00H, 00H, 00H, 00H ; 0FC8 _ o....... db 34H, 00H, 00H, 00H, 03H, 00H, 00H, 00H ; 0FD0 _ 4....... db 0CH, 00H, 01H, 00H, 10H, 00H, 01H, 00H ; 0FD8 _ ........ db 00H, 00H, 00H, 00H, 00H, 00H, 00H, 00H ; 0FE0 _ ........ SECTION ._DATA.__nl_symbol_ptr align=8 noexecute ; section number 5, data imp_dyld_stub_binder: ; import from ? dq 0000000000000000H ; 1000 _ 0000000000000000 imp__exit: ; dword dd 00000000H ; 1008 _ 0 imp__write: ; dword dd 00000000H ; 100C _ 0 SECTION ._DATA.__la_symbol_ptr align=8 noexecute ; section number 6, data ?_005: ; switch/case jump table dq 0000000100000F8CH ; 1010 _ 0000000100000F8C ?_006: ; switch/case jump table dq 0000000100000F96H ; 1018 _ 0000000100000F96


Code:
Dump of file: mac_test, type: Mach-O Little Endian64 Dump of Mach-O file mac_test ----------------------------------------------- File size: 0x2120 File header: CPU type: Intel 64 bit, subtype: unknown(0x80000003) File type: Mach-O Little Endian - demand paged executable file Number of load commands: 15, Size of commands: 0x460, Flags: 200085 Command 1: 64-bit segment, size: 0x48 Name: __PAGEZERO, Memory address 0x0000000000000000, Memory size 0x0000000100000000 File offset 0x0000000000000000, File size 0x0000000000000000 Maxprot 0x0, Initprot 0x0 Number of sections 0, Flags 0x0 Command 2: 64-bit segment, size: 0x188 Name: __TEXT, Memory address 0x0000000100000000, Memory size 0x0000000000001000 File offset 0x0000000000000000, File size 0x0000000000001000 Maxprot 0x7, Initprot 0x5 Number of sections 4, Flags 0x0 Command 3: 64-bit segment, size: 0xE8 Name: __DATA, Memory address 0x0000000100001000, Memory size 0x0000000000001000 File offset 0x0000000000001000, File size 0x0000000000001000 Maxprot 0x7, Initprot 0x3 Number of sections 2, Flags 0x0 Command 4: 64-bit segment, size: 0x48 Name: __LINKEDIT, Memory address 0x0000000100002000, Memory size 0x0000000000001000 File offset 0x0000000000002000, File size 0x0000000000000120 Maxprot 0x7, Initprot 0x1 Number of sections 0, Flags 0x0 Command 5: unknown(0x80000022), size: 0x30 Command 6: Symbol table, size: 0x18 Symbol table offset 0x2078, number of symbols 5, String table offset 0x20E0, String table size 0x40 Command 7: dynamic link-edit symbol table info, size: 0x50 Index to local symbols 0, number of local symbols 0, Index to external symbols 0, number of external symbols 2, Index to undefined symbols 2, number of undefined symbols 3, File offset to TOC 0x0, number of entries in TOC 0, File offset to module table 0x0, Number of module table entries 0, Offset to referenced symbol table 0x0, Number of referenced symtab entries 0 Offset to indirect symbol table 0x20C8, Number of indirect symtab entries 6 Offset to external relocation entries 0x0, Number of external reloc. entries 0 Offset to local relocation entries 0x0, Number of local reloc. entries 0 Command 8: load a dynamic linker, size: 0x20 Command 9: uuid, size: 0x18 Command 10: unknown(0x24), size: 0x10 Command 11: unknown(0x2A), size: 0x10 Command 12: unknown(0x80000028), size: 0x18 Command 13: load a dynamicly linked shared library, size: 0x38 Command 14: unknown(0x26), size: 0x10 Command 15: unknown(0x29), size: 0x10 Sections: Section 1: Name: _code, Segment: __TEXT. Memory address 0xF40, Size 0x2E, File offset 0xF40 Alignment 64, Reloc. ent. offset 0x0, Num reloc. 0 Flags 0x80000400, reserved1 0x0, reserved2 0x0 Section 2: Name: __stubs, Segment: __TEXT. Memory address 0xF6E, Size 0xC, File offset 0xF6E Alignment 2, Reloc. ent. offset 0x0, Num reloc. 0 Flags 0x80000408, reserved1 0x0, reserved2 0x6 Section 3: Name: __stub_helper, Segment: __TEXT. Memory address 0xF7C, Size 0x24, File offset 0xF7C Alignment 4, Reloc. ent. offset 0x0, Num reloc. 0 Flags 0x80000400, reserved1 0x0, reserved2 0x0 Section 4: Name: __unwind_info, Segment: __TEXT. Memory address 0xFA0, Size 0x48, File offset 0xFA0 Alignment 4, Reloc. ent. offset 0x0, Num reloc. 0 Flags 0x0, reserved1 0x0, reserved2 0x0 Section 5: Name: __nl_symbol_ptr, Segment: __DATA. Memory address 0x1000, Size 0x10, File offset 0x1000 Alignment 8, Reloc. ent. offset 0x0, Num reloc. 0 Flags 0x6, reserved1 0x2, reserved2 0x0 Section 6: Name: __la_symbol_ptr, Segment: __DATA. Memory address 0x1010, Size 0x10, File offset 0x1010 Alignment 8, Reloc. ent. offset 0x0, Num reloc. 0 Flags 0x7, reserved1 0x4, reserved2 0x0 Symbol table: Public symbols: 0 __mh_execute_header, Section 1, Value 0x0 External, Defined Reference type: External non lazy, Flags: Referenced dynamically, 1 _main, Section 1, Value 0xF40 External, Defined Reference type: External non lazy, Flags: External symbols: 2 _exit, Section 0, Value 0x0 External, Undefined, no section Reference type: External non lazy, Flags: 3 _write, Section 0, Value 0x0 External, Undefined, no section Reference type: External non lazy, Flags: 4 dyld_stub_binder, Section 0, Value 0x0 External, Undefined, no section Reference type: External non lazy, Flags: Indirect symbols: _exit, type 0x1, sect 0, desc 0x100, val 0x0 _write, type 0x1, sect 0, desc 0x100, val 0x0 dyld_stub_binder, type 0x1, sect 0, desc 0x100, val 0x0 Unknown(0x40000000) _exit, type 0x1, sect 0, desc 0x100, val 0x0 _write, type 0x1, sect 0, desc 0x100, val 0x0 String table: 0: 2: __mh_execute_header 22: _main 28: _exit 34: _write 41: dyld_stub_binder 58: 59: 60: 61: 62: 63:


Description: working binary - remove .txt extension
Download
Filename: mac_test.txt
Filesize: 8.28 KB
Downloaded: 53 Time(s)

Post 21 Aug 2017, 08:14
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6862
Location: Kraków, Poland
tthsqe wrote:
Thomas, your 64 macros don't work for dynamic linking.
Yes, I mentioned that already. The addresses of functions are resolved correctly but jumping to them causes segfault, because dynamic linker does not map the libraries as executable memory. This may have something to do with the expectation that functions should be linked as "lazy" externals, while my example (made in ELF-like fashion) uses non-lazy linking for them. Perhaps the dynamic linker then assumes that these are data symbols instead of functions and does not give them executable rights. For some reason this worked with 32-bit, but perhaps the NX bit is not used there.

The "lazy" externals require additional "trampolines" as you mentioned, but I did not find out what makes them tick (I like to know exactly how things work and then make the absolutely minimal implementation). Each stub (the one that consists only of a single JMP instruction) has to initially jump to the trampoline, and that code in turn has to call the dynamic linker somehow, and provide the parameters (in registers?) telling what function needs to have it address resolved and updated in the stub - this is the part I have not yet figured out. It make take some time before a have a chance to work on it more - if you find out anything in the meantime, please let me know.
Post 21 Aug 2017, 10:51
View user's profile Send private message Visit poster's website Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 714
Ok. This is probably take a long time to work out. I can see right away that for 64bits, your macros are generating jmp instructions in a __jmp_table section. It looks like the way it is done is to put a "dq ?" in the __DATA,__nl_symbol_ptr section.

I'm reading https://www.mikeash.com/pyblog/friday-qa-2012-11-09-dyld-dynamic-linking-on-os-x.html
Post 21 Aug 2017, 13:54
View user's profile Send private message Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 283
Location: Australia
I attempted to create a VirtualBox guest out of macOS Sierra yesterday to put up, the VM gets stuck waiting on the root device though so I was unable to finish.

Didn't have a ton of time to research, but if any of you have ideas to get me past that point I can donate another macOS VM to play around with.

_________________
2 Ton Digital - https://2ton.com.au/
Post 21 Aug 2017, 21:23
View user's profile Send private message Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 6862
Location: Kraków, Poland
tthsqe wrote:
Ok. This is probably take a long time to work out. I can see right away that for 64bits, your macros are generating jmp instructions in a __jmp_table section. It looks like the way it is done is to put a "dq ?" in the __DATA,__nl_symbol_ptr section.
What my macros generate is the non-lazy binding, and it is actually quite similar to dynamic linking of ELF executables or PE imports. In all cases you have a variable that is updated by dynamic linker with the pointer to a function at run-time:
Code:
_exit dq ? ; DD in 32-bit case
You can then simply call the function by using this pointer directly (and the direct use of pointer requires an indirect call):
Code:
call [_exit]
and this is in fact the default way fasm's macros are wired for ELF and PE. In all cases, including ELF and PE, you can also define additional function stubs and call them instead:
Code:
call exit ; ... exit: ; stub jmp [_exit]
In ELF and PE this is something you can use optionally, it is not forced. In Mach-O for some reason dynamic linking does not work properly unless you have these stubs.

OK, we now have non-lazy binding covered, not let's talk about the lazy one, which may actually be necessary for the functions in 64-bit Mach-O (though this is just my theory at the moment). In this case dynamic linker does not fill the import variables with pointers to the functions. These variables should be pre-filled at the assembly time with pointers to the "stub helpers":
Code:
_exit dq _exit.helper _exit.helper: ; some code that calls dynamic linker and tells it to update [_exit] with a pointer to the external function, and then jump there
The helper gets called only once, the first time the function is called, because then the variable get updated with the resolved pointer to the function. Note that this is independent from whether you call the function through the stub or directly.

Now the detail I was missing when I wanted to change my macros from non-lazy binding to a lazy one is what that stub helper should do, exactly (in both 32-bit and 64-bit variants) and how can it call the dynamic linker.
Post 22 Aug 2017, 19:33
View user's profile Send private message Visit poster's website Reply with quote
Grom PE



Joined: 13 Mar 2008
Posts: 113
Location: i@grompe.org.ru
redsock wrote:
I attempted to create a VirtualBox guest out of macOS Sierra yesterday to put up, the VM gets stuck waiting on the root device though so I was unable to finish.

Didn't have a ton of time to research, but if any of you have ideas to get me past that point I can donate another macOS VM to play around with.
I was able to run some preinstalled image of macOS El Capitan on VMWare.

But an image of macOS Sierra didn't want to cooperate — it crashed (rebooted) in minutes after any nontrivial action, and fasmg and related binaries gave an error "Illegal Instruction: 4" on it. I'm not sure if I got a broken image of Sierra or its loader is more strict and something is missing. Or maybe something to do with the fact I'm trying this on AMD processor.
Post 23 Aug 2017, 05:15
View user's profile Send private message Visit poster's website Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 714
Thomasz, there is the source I am using.

Code:
include 'x86/include/x64.inc' use64 MachO.Settings.ProcessorType equ CPU_TYPE_X86_64 MachO.Settings.BaseAddress = 0x100000000 include 'x86/macinc/macho.inc' entry Start interpreter '/usr/lib/dyld' uses '/usr/lib/libSystem.B.dylib' (1.0.0, 1.0.0) import _write,'_write' import _exit,'_exit' segment '__TEXT' readable executable Start: and rsp, -16 mov edi, 1 lea rsi, [msg] mov edx, msg_end - msg call _write mov edi, eax call _exit msg: db 'Hello World!',10 msg_end:


your current macros are generating something like this
Code:
segment '__TEXT' readable executable Start: and rsp, -16 mov edi, 1 lea rsi, [msg] mov edx, msg_end - msg call _write mov edi, eax call _exit msg: db 'Hello World!',10 msg_end: segment '__IMPORT' readable writable executable section '__nl_symbol_ptr' _write2: dq ? _exit2: dq ? section '__jump_table' _write: jmp qword[_write2] _exit: jmp qword[_exit2]



The problem with this is that dyld is going to have to write the correct addresses
into qword[_write2] and qword[_exit2]. But this segment is marked executable. I though this was a big no-no.
This might explain why your current macros generate exe's that segfault in dyld.


From what I can tell you have to do something like this
Code:
segment '__TEXT' readable executable Start: and rsp, -16 mov edi, 1 lea rsi, [msg] mov edx, msg_end - msg call qword[_write] mov edi, eax call qword[_exit] msg: db 'Hello World!',10 msg_end: segment '__DATA' readable writable section '__nl_symbol_ptr' _write: dq ? _exit: dq ?

and direct dyld to fill in qword[_write] and qword[_exit]
Post 23 Aug 2017, 07:19
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >

Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 2004-2018, Tomasz Grysztar.

Powered by rwasa.