x86 convert flags from unsigned cmp to flags from signed cmp

Index > Main > x86 convert flags from unsigned cmp to flags from signed cmp

Author

Thread

tthsqe

Joined: 20 May 2009
Posts: 767

tthsqe 13 Jun 2023, 01:55

I haven't seen this one here before. How would you convert the flags from an 'unsigned' cmp to the corresponding 'signed' flags. What I mean is: suppose we start with the zf and cf and ignore the sf and of. How would you preserve zf and ensure that the new (sf xor of) matches the old cf. This makes the following table of conversions in the condition codes:

Code:

 ja =>  jg
jae => jge
 jb =>  jl
jbe => jle

Elegant answers desired. Answers to inverse problem welcome as well.

13 Jun 2023, 01:55

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20451
Location: In your JS exploiting you and your system

revolution 13 Jun 2023, 02:13

LAHF & SAHF.

Use a 256 byte table:

Code:

lahf
movzx eax,ah
mov ah,[eax+conversion_table]
sahf

conversion_table:
 db ...

13 Jun 2023, 02:13

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 13 Jun 2023, 13:54

The only problem I see with the LAHF/SAHF table solution is that overflow flag is undefined, but it could be fixed with SAR AL, 1.

I haven't tested these, but I think this solves the setup for the signed branch (it's a little tricky):

Code:

setnc ah
setnz al
ror ah, 1
shl ax, 1

... and then the setup for unsigned branch (I think there is a simpler solution as this is very literal):

Code:

lahf
seto al         ; there are smaller solutions
ror al, 1       ; putting overflow flag in bit 7
xor al, ah
sahf
bt ax, 7        ; CF = OF xor CF

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

13 Jun 2023, 13:54

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 13 Jun 2023, 14:40

Interesting exercise. My attempt:

Code:

        sets    al
        rcr     al,2
        rol     al,1

These instructions keep ZF and SF unchanged. RCR sets up the highest two bits of AL with values of SF and CF. Final ROL calculates OF as xor of these two bits, therefore we have OF = (old CF xor SF), which gives us OF xor SF = old CF. Am I right?

And the other direction seems trivial:

Code:

        setl    al      ; LSB(AL) <- SF xor OF
        rcr     al,1    ; CF <- LSB(AL), ZF unaffected

13 Jun 2023, 14:40

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 13 Jun 2023, 15:48

Tomasz Grysztar wrote:

My attempt:

Code:

        sets    al
        rcr     al,2
        rol     al,1

Am I right?

... seems to fail with CMP $7F, $FF flags, SF=OF=1. Your conversion produces SF<>OF.

My solution is incorrect as well.

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

13 Jun 2023, 15:48

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 13 Jun 2023, 16:18

bitRAKE wrote:

... seems to fail with CMP $7F, $FF flags, SF=OF=1. Your conversion produces SF<>OF.

The code is supposed to set up flags so that SF xor OF = old CF.

After your CMP there is CF = 1, SF = 1, conversion sets up OF = 0, so (SF xor OF) = (1 xor 0) = 1, which is correct because this was the value of CF.

13 Jun 2023, 16:18

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 13 Jun 2023, 16:41

Ah, I got my test cases mixed up. It works now, and I can fix my solution to:

Code:

        setc ah
        setnz al
        ror ah, 1
        shl ax, 1

... using rotates to preserve the Z-flag is definitely the way to go.

This is what I use to verify (I've named them quite verbosely, now.):

Code:

macro GATHER_STATES_SIGNED ; need to preserve S<>O, Z
        pushfq
        sets al
        seto ah
        cmp al, ah
        setnz ah
        popfq
        setz al
        shl ah,1
        or al,ah
end macro
macro GATHER_STATES_UNSIGNED ; need to preserve C, Z
        setz al
        setc ah
        shl ah,1
        or al,ah
end macro

        mov ecx, 0x10000
@@:     cmp cl, ch
        GATHER_STATES_UNSIGNED ; need to preserve C, Z
        mov dl, al

        cmp cl, ch
        SOLUTION
        GATHER_STATES_SIGNED ; need to preserve S=O, Z
        cmp dl, al
        loopz @B

... only need to swap the *_SIGNED/*_UNSIGNED macro usage to test the reverse goal.

13 Jun 2023, 16:41

tthsqe

Joined: 20 May 2009
Posts: 767

tthsqe 14 Jun 2023, 01:01

Wow, so apparently there is a lahf and a sahf instruction, and the rotate instructions leave both ZF and SF unchanged. I would shorten revolution's LUT to 4 bytes

Code:

setz cl
adc cl,cl
shl cl,3
mov eax, the right 4 bytes
ror eax,cl
sahf

But, yes, there is the problem with OF, so what is the point of sahf if OF is undefined (or is it just unaffected?)? Thomasz' solution seems to take the cake.

By the way, I also realized that only 7 of the 16 possible combinations of ZF,CF,SF,OF are possible after a cmp instruction, though I don't think any of the solutions assume, for example, that ZF=CF=1 is impossible. However, bitRAKE's test harness won't generate this input flag combination(?).

14 Jun 2023, 01:01

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 14 Jun 2023, 01:22

tthsqe wrote:

though I don't think any of the solutions assume, for example, that ZF=CF=1 is impossible. However, bitRAKE's test harness won't generate this input flag combination(?).

I think ZF=CF=1 should be possible - which means I need to expland the test harness. I was just throwing something together in my normal lazy fashion - the longer I think about it - the more I would do it differently. Very Happy

(One more abstraction layer and I should be able to make a completely general test.)

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

14 Jun 2023, 01:22

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 14 Jun 2023, 03:51

Code:

; RSI : expectations
; EDX : source flag mask
; EBX : destination flag mask
        popcnt eax, edx
        xor ecx, ecx
        bts ecx, eax
.checks:
        ; generally update masked flags based on present iteration
        pdep eax, ecx, edx
        pushfq
        or dword [rsp], edx
        xor dword [rsp], edx
        or dword [rsp], eax
        popfq

        SOLUTION

        pushfq
        pop rax
        pext eax, eax, ebx
        cmp [rsi + (rcx-1)*4], eax
        loopz .checks

... better test case(s). Just pre-calculate the result table based on equations using source bit values, and exclude cases not happening in your code. (For example, CMP cannot produce ZF=CF=1.)

Note: the CMP at the end is not sufficient as we really want to match equations of result bits.

14 Jun 2023, 03:51

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum