Advanced ALIGN (for fasmg)

Index > Macroinstructions > Advanced ALIGN (for fasmg)

Author

Thread

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 01 Feb 2019, 17:29

Note: this has been split off a thread about alignments in "struct" macro for fasm 1.

Redefining ALIGN to be relative to the "SCALE 0" component would be extremely useful, and doesn't appear to lose any generality.

Personally, I've always wanted ALIGN to take arguments like "3 MOD 16" because there are code cases that just work better that way. Doing a separate ALIGN 16 and three NOPs doesn't have the same effect because sometimes only 0-2 nops are needed. Being able to define a custom solution seems to be the best way.

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

01 Feb 2019, 17:29

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 02 Feb 2019, 09:22

bitRAKE wrote:

Redefining ALIGN to be relative to the "SCALE 0" component would be extremely useful, and doesn't appear to lose any generality.

Personally, I've always wanted ALIGN to take arguments like "3 MOD 16" because there are code cases that just work better that way. Doing a separate ALIGN 16 and three NOPs doesn't have the same effect because sometimes only 0-2 nops are needed. Being able to define a custom solution seems to be the best way.

This is why in the design of fasmg I decided to not include ALIGN and leave it to specialized macro implementation. There are so many possible factors.

The basic implementation in fasm tries to align the actual resulting absolute values, so if the value contains some variable term (analogous to ELEMENT terms in fasmg), it is only possible to align up to value that we know this variable is aligned to. For example, if variable term is the base address of section (in object formats), we know that this address is aligned as set up by section header and it is not possible to align better than that. If variable term is a register, we do not know anything about its alignment, so fasm cannot do much (perhaps something like "assume ebx aligned to 4" could help - again, something that can be easily prototyped with macros).

I'm also briefly talking about these alignment issues in my PE tutorial:

Quote:

According to the specification, the address of an image should always be a multiple of 65536. Therefore the addresses aligned to any power of two up to that number should stay aligned when the base is moved.

Last edited by Tomasz Grysztar on 04 Feb 2019, 13:26; edited 1 time in total

02 Feb 2019, 09:22

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 02 Feb 2019, 10:52

And because playing with fasmg can in general improve my mood, I prepared an example of rich-featured ALIGN package:

Code:

define align?

align?.INDEX = 0

macro align?.assume variable, pow2
        assert bsf(pow2) = bsr(pow2)
        repeat align?.INDEX+1
                if % = %%
                        align?.INDEX = align?.INDEX + 1
                        align?.v% = variable
                        align?.% = pow2
                else if align?.v% relativeto variable
                        align?.v% = variable
                        align?.% = pow2
                        break
                end if
        end repeat
end macro

macro align? pow2*, remainder:0, filler:?
        local offset
        offset = $
        repeat align?.INDEX
                if offset relativeto align?.v%
                        if align?.% >= pow2
                                offset = offset - align?.v%
                        else
                                err 'variable portion of address is not aligned enough'
                        end if
                end if
        end repeat
        assert bsf(pow2) = bsr(pow2)
        db (-offset+remainder)and(pow2-1) dup filler
end macro

You can use it like:

Code:

align.assume ebp+4, 16  ; assume that ebp+4 is aligned to 16 bytes

virtual at ebp-50 as 'test'
        var0 db 0
        align 16        ; var1 is aligned under the assumption
        var1 dd 1
        align 16, 3     ; this makes var2-3 aligned
        var2 db 2
end virtual

02 Feb 2019, 10:52

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 02 Feb 2019, 19:03

Just to be pedantic on this topic: I'll mention that some use a patterned fill within their software - as a type of signature/security watermark.

The flexibility is virtually endless.

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

02 Feb 2019, 19:03

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 02 Feb 2019, 19:07

Yeah, and there are also the various length NOP instructions for code alignment...

02 Feb 2019, 19:07

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 03 Feb 2019, 01:32

EMIT makes the multibyte NOPs very simple:

Code:

virtual at 0
NOPS::
        emit 1: $90
        emit 2: $6690 bswap 2
        emit 3: $0f1f00  bswap 3
        emit 4: $0f1f4000  bswap 4
        emit 5: $0f1f440000 bswap 5
        emit 6: $660f1f440000 bswap 6
        emit 7: $0f1f8000000000 bswap 7
        emit 8: $0f1f840000000000 bswap 8
        emit 9: $660f1f840000000000 bswap 9
        emit 10:$66660f1f840000000000 bswap 10
        emit 11:$6666660f1f840000000000 bswap 11
        emit 12:$6666662e0f1f840000000000 bswap 12
        emit 13:$666666662e0f1f840000000000 bswap 13
        emit 14:$66666666662e0f1f840000000000 bswap 14
        emit 15:$6666666666662e0f1f840000000000 bswap 15
end virtual

macro nopn? n*
        if n < 0
                err "nopn expects length of NOP instruction"
        else
                repeat 1,d16:n/16,dm:n mod 16
                        repeat d16
                                ; invalid instruction
                                emit 16:$666666666666662e0f1f840000000000 bswap 16
                        end repeat
                        if dm > 0
                                load dat:dm from NOPS: (dm*(dm-1))/2
                                emit dm:dat
                        end if
                end repeat
        end if
end macro

Patterned fill can obviously be realized in a similar manner.

Used https://stackoverflow.com/questions/25545470/long-multi-byte-nops-commonly-understood-macros-or-other-notation as NOP reference.

Just occurred to me that 16 bytes is too long, should change to just MOD 15.

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

03 Feb 2019, 01:32

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 03 Feb 2019, 08:18

And only a very small tweak is needed in my above ALIGN macro to make it work with custom fillers like that:

Code:

macro align? pow2*, remainder:0, filler:rb
        local offset
        offset = $
        repeat align?.INDEX
                if offset relativeto align?.v%
                        if align?.% >= pow2
                                offset = offset - align?.v%
                        else
                                err 'variable portion of address is not aligned enough'
                        end if
                end if
        end repeat
        assert bsf(pow2) = bsr(pow2)
        filler (-offset+remainder)and(pow2-1)
end macro

Now you can simply give NOPN as a third argument.

It is interesting to use a custom filler when aligning sections in PE file. In the PE template I made in my tutorial it is enough to alter a single line in the SECTION macro (after replacing its simple ALIGN macro with the set of macros produced here, of course):

Code:

                align FILE_ALIGNMENT,,nopn

When I started thinking about possible patterns to use there, I had one funny idea. Imagine that you lay out your binary file on top of another one and treat the alignment areas as transparent bits, so that you can see parts of the other file in between your data. I implemented it like this:

Code:

define alpha0?

macro alpha0?.set? filepath*
        local area
        define alpha0?.data area
        virtual at 0
                area:: file filepath
                alpha0?.size = $
        end virtual
end macro

macro alpha0?.fill? n*
        local pattern,length
        length = n
        if $% + length > alpha0?.size
                length = alpha0?.size
        end if
        if length
                load pattern:length from alpha0?.data:$%
                db pattern
        end if
        db n-length dup 0
end macro

and then in PE example I have set it up to use its own source as the underlying file:

Code:

alpha0.set __FILE__

and then changed the file alignment to use it:

Code:

                align FILE_ALIGNMENT,,alpha0.fill

But why? Just for fun!

03 Feb 2019, 08:18

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 03 Feb 2019, 09:32

Another kind of filler - one that duplicates already existing code or data:

Code:

define photocopy?

macro photocopy?.set?
        local area
        define photocopy?.data area
        area::
        photocopy?.origin = $$
        photocopy?.size = $@ - $$
end macro

macro photocopy?.make? n*
        local length,pattern
        length = n
        if photocopy?.size
                while photocopy?.size <= length
                        if % = 1
                                load pattern:photocopy?.size from photocopy?.data:photocopy?.origin
                        end if
                        db pattern
                        length = length - photocopy?.size
                end while
                load pattern:length from photocopy?.data:photocopy?.origin
                db pattern
        else
                db length dup 0
        end if
end macro

In the SECTION macro in my tutorial it can be used like:

Code:

                photocopy.set
                section $%%
                align FILE_ALIGNMENT,,photocopy.make

And it then just repeats the contents of the section to fill the alignment area. This one is very compression-friendly. Turns out the resulting file may compress even better than one with a zero-filled alignment!

03 Feb 2019, 09:32

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 04 Feb 2019, 08:25

Another fun idea - first a simple filler macro:

Code:

macro fill? params*
        match {s} n, params
                local bytes,length
                bytes = string s
                length = n
                while lengthof bytes <= length & lengthof bytes > 0
                        db bytes
                        length = length - lengthof bytes
                end while
                if length > 0
                        emit length : bytes and (1 shl (length*8) - 1)
                end if
        else
                err 'unexpected syntax'
        end match
end macro

and then in a SECTION macro alignment set up like this:

Code:

                if previous
                        align FILE_ALIGNMENT,,fill {Section.previous.NAME}
                else
                        align FILE_ALIGNMENT,,fill {':HEADERS'}
                end if

This nicely demarkates all sections in PE file so that when you look at it in hex viewer you immediately see boundaries of all its areas.

04 Feb 2019, 08:25

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 04 Feb 2019, 09:50

I forgot to mention, to make this ALIGN work correctly with relocatable variants of PE templates from my tutorial all that is needed is this line:

Code:

align.assume IMAGE_BASE, 10000h

It really gives us some nice semantics.

04 Feb 2019, 09:50

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 01 Feb 2020, 20:14

I have rewritten the universal ALIGN macro using CALM and published it on GitHub (in the general utility package). It has a slightly modified syntax - the are only two arguments, alignment and filler, and the offset (remainder) can be specified in the first argument with a "|" separator. This allows to avoid double commas that I had to use in the above samples. After correcting for that change, they all should work with the new implementation.

01 Feb 2020, 20:14

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum