Hi guys, I'm not a compiler or assembler developer so pardon me if this sounds silly .

I found this rather weird C-compiled executable at work, but it seems to use mixed calling convention within the generated x86 (16-bit [maybe big-real]) code. What I observed is the calling convention mixed between stdcall and fastcall. I suspect it's generated by Watcom 16-bit compiler (Open Watcom??). Not so sure about it.

Anyone have experience on this? or probably have been working on things like this before.

Well, I didn't have a computer during the DOS 4GW era, so I don't have lots of experience with 16-bit big-real mode or even the "standard" 16-bit mode.

Hm, what is the problem with the mixed calling convention? I use it all the time, because of different reasons.

Actually, not a problem. But, I want to know whether 16-bit compiler(s) could emit code with different calling conventions? I have very little exposure to 16 compilers like Watcom C/C++ or Open Watcom.

I don't know about C/C++. The code written by me on 16 bit computers was in Turbo Pascal and TASM assembler.
IIRC, both supports different calling conventions.

Same here, but I can think of a few reasons:
1. The programmer(s) explicitly specified the calling convention for each function (or some of them and the rest established by the compiler by default)
2. The stdcall (not cdelc??) functions are not marked static and therefore the compiler's optimization engine felt free to change the calling convention to an optimal one for the functions that could result beneficial (sure it was exactly fascall and not really a free register-based convention?)
3. Besides using the same logic as in to 2., also some non-static functions were internally duplicated by the compiler to be able to change the convention (this is an available option in cases where the callee and caller are in the same source file)

If the compiler was capable of doing 3 or 2, then in the disassembly you may be missing some functions because they end up inlined even if the programmer(s) didn't specify inline explicitly.

hi,

about 1: Perhaps some of the code I observed are mixed assembly+C. The thing is bx register is sometimes used to pass parameters while in most cases, the parameters are passed via stack. Or is it possible that these "outlier" functions are coded to use __fastcall??

about 2: I can say I'm 90% sure it's stdcall calling convention because mostly the callee cleans-up the stack.

about 3: I see. I have never used such feature so I didn't know it previously. Well, thanks for pointing out .

Borland C++ Compiler used __fastcall for the VCL(visual component library). Because of the VCL was writen in Delphi Pascal.

So this way, VCL function like TButton1::Clik was __fastcall while the procedure you writen where cdecl. And then, if you called some winapi function, it would be defined as extern stdcall.

Thanks Enko. That's insightful

(I'm necroing here, but I came here by chance via google).

Some 16-bit compilers like Topspeed's 3.x line (having a C/C++, Pascal and Modula2 compiler) could handle custom calling conventions. So you could specify per function if first parameter should go into ax, second into bx, third on the stack and fourth in dx etc.

That is mainly made to optimize (e.g. with an interface to the out port instruction as prime example) or 3rd party libraries with atypical calling conventions.

Sounds cool, I wish GCC had something like that builtin without requiring some plugin extension.

The GCC devs will complain that it isn't portable.

What is AX on ARM/MIPS?

They could just ignore it on another platform; it already has inline asm and that won't even compile in such case. I know what you mean but yeah they're just dumb.