flat assembler
Message board for the users of flat assembler.

Index > Main > Avx VPCMPB error show Ida pro

Author
Thread Post new topic Reply to topic
Roman



Joined: 21 Apr 2012
Posts: 1794
Roman 15 Aug 2023, 10:07
Code:
;fasmw 1.73 compiled fine
;32 bits program
VPCMPB k1 {k2}, ymm2, ymm3, 1
    

But Ida Pro show error: execute an illegal instruction
If I run my program I get crash.
Amd Ryzen 3500 avx supported
Post 15 Aug 2023, 10:07
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20342
Location: In your JS exploiting you and your system
revolution 15 Aug 2023, 10:13
AFAICT fasm produces the correct encoding.
Code:
00000000  62F36D2A3FCB01    vpcmpltb k1{k2},ymm2,ymm3    
Post 15 Aug 2023, 10:13
View user's profile Send private message Visit poster's website Reply with quote
Roman



Joined: 21 Apr 2012
Posts: 1794
Roman 15 Aug 2023, 10:15
VPCMPB this is avx 512
Maybe my ryzen 5 3500 not supported
Post 15 Aug 2023, 10:15
View user's profile Send private message Reply with quote
Roman



Joined: 21 Apr 2012
Posts: 1794
Roman 15 Aug 2023, 10:18
Quote:
vpcmpltb k1{k2},ymm2,ymm3


Fasmw 1.73 compiled fine
But Ida Pro show
VPCMPB k1{k2}, ymm2, ymm2, 1
Post 15 Aug 2023, 10:18
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20342
Location: In your JS exploiting you and your system
revolution 15 Aug 2023, 10:23
I used the nasm disassembler. I don't vouch for the correctness of the disassembly, but it shows the 7 bytes produced correctly.
Post 15 Aug 2023, 10:23
View user's profile Send private message Visit poster's website Reply with quote
Roman



Joined: 21 Apr 2012
Posts: 1794
Roman 15 Aug 2023, 10:42
Ok. Forget
I found
vpcmpeqb ymmdest, ymmsrc1, ymmsrc2
vpcmpgtb ymmdest, ymmsrc1, ymmsrc2

Ymmdest this is mask to YMM reg

All work fine in 32 bits
Post 15 Aug 2023, 10:42
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.