compile "fasmd.asm" and compare size "fasmd.exe" it with original
see changes in "update.inc"

fasm does not generate a 66h-prefixed jumps unless explicitly told so (with form like "jmp near word"). If you look at the sources of very early versions (0.9) of fasm you may notice that it did initially optimize conditional jumps this way, but I later changed it - after I learned that prefixes may actually hurt performance. Nowadays I consider this to be one of the controversial optimizations that I decided not to include as a default behavior, I think they should be chosen consciously by the programmer.

It's also important to remember that $66 makes a relative jump not completely base independent (base range dependent), which means a functional difference for some cases. I.e. 66 0F 84 80 00 may jump to a location different from where 0F 84 80 00 00 00 jumps.

That alone is all that is needed to disqualify it as a potential optimisation. Any "optimisation" with a functional difference is not a real optimisation, but instead it is a timebomb waiting to go off.

This optimization (jmp word 0x?) is turned off. (see jmp_address=1) I use it in unreal mode.

Try compile "fasmd.asm" attached and compare files.

CandyMan

Well, I got two identical files. And the standard fasm compiles the following:

into a 2-bytes opcode.
revolution

I agree with you, but fasm in fact does so called "aggressive" optimizations that may result in functional difference, which I'm not happy about. The point is however that there are assumptions that can legally be made explicit, and the optimization is then allowed to rely on these. E.g., you assume a flat model, or you assume the base dependency. But the assumptions should be made very carefully to not violate the general programmer's expectations such as that relative jumps are base independent.

I am insisting in addition that, the jump ahead is badly calculated as near instead of short.

CandyMan
Please, provide a minimal complete source allowing to reproduce the problem.

it will be sufficient to add these lines to the code:

I think that what he means is actually related to the question you asked recently. When an expression contains some values that fasm sees for the first time, the complete value of expression is assumed to be zero - and the same applies to relative offsets used by jumps. This way in the first pass the instructions that may get optimized (not only jumps, but also instructions like "add eax,forward_referenced_constant") have zero predicted for their immediates when no better prediction is possible. Generating short forms in the initial pass helps to find a more compact solution.

The suggestion made by CandyMan is that in the very first pass short forms of instructions should be enforced even when the values are already known (like in the case of a jump backwards). And yes, it possible that this could help fasm to find a bit better solution, because these value can still change, and there is some probability that such value may in the next pass change in such a way, that the short form of instruction will stay.

I've just tried such change in the general mechanism - right at the beginning of "calculate_expression", where it sets up [value_undefined] as 0, I added a code that sets it to 1 in the very first pass. This way in the first pass the value of every expression is perceived to be zero. And yes, it does assemble fasmd.exe a four bytes shorter then. But at the same time it is dangerous and may cause some other things to stop working. On the other hand, using a less general mechanism, like proposed tweaking of the jump handler, is probably safe.

Any changes to the prediction methods are in fact a double-edged sword, I mentioned it one or two times already - any such tweak may make some types of sources to assemble into a smaller code, while breaking some other ones in such a way that they get assembled into a larger output or refuse to assemble at all. In the case of "fasm 1" I obviously choose the methods that focus on assembling the standard x86 code well (and they are not always working well with other things) - the zeroing of undefined expressions was created with this in mind. So perhaps this also could be a good modification - to make all such optimized instructions (not only jumps) have short form in the very first pass. But note that this still would be an additional mechanism, not a replacement of existing one - because zeroing of expressions has an effect not only in the first pass, but in the later ones too, if you have some dependencies that cause symbols to get defined only in later passes, etc.

It is all a bit of a heuristic, that should be chosen based on the results obtained for "the most typical" source codes. I recall that I tested some similar modification long time ago and at the time it did not make any difference, so I scrapped the idea. Now I have an example of a four-byte difference. Is it worth it then? Possibly.

Tomasz, thank you for the exhaustive reply.

Tomasz Grysztar
Are you sure, it's about predicting zeroes? As long as CandyMan failed to respond to my previous post, I'm doing it myself now:

A long form of the jump is compiled here because the increase in the jump size at the second pass is critical itself for the choice of the jump encoding. The critical difference is initiated by mispredicting fwdref to be a dword instead of a byte at the first pass and this way shifting the address of the jump at the second pass. Thus maybe it makes more sense to also predict shorter forms for instructions with encodings having nothing to do with predicting zeroes. Intuitively this would be a less of black magic solution.

Tomasz Grysztar
I just realized that I actually repeated your suggestion. I was a bit confused by your example of "not only jumps" being add eax,forward_referenced_constant, and I thought that "all such optimized instruction" still referred solely to the instructions optimized by initially predicting a zero.

Your example is quite important, though, because it's something I overlooked and that should be corrected right away. When a size of label is predicted to be 0 (which registers an "operand size not specified" recoverable error) the immediate should better be assumed to be 8-bit instead of 32-bit (and this, like the zeroing of expressions, is something that is independent from the pass number). I did it in some places (like the "basic_mem_imm" handler), but apparently I overlooked it in some other (like the "mov_mem_imm" you pointed out) - so it may be considered a kind of bug, actually.

Tomasz Grysztar
Well, then you may want to fix another problem that may go unnoticed because of this fix. With the version 1.71.26 even enforcing a short jump didn't work:

It doesn't seem reasonable to fallback to the larger offset in case the short keyword is specified. It is interesting however that defining a label at the jump location magically solved both problems making the mov be predicted in the short form from the very beginning:

This compiles into an optimal form with 1.71.26. This might reveal a solution for a bit wider range of optimization problems.

This is where the "label adjustment" mechanism comes into play. By defining a label you make assembler notice what is the "shift" between it value from previous pass and current one, and it takes this offset into consideration when predicting the value of next label.

Tomasz Grysztar
That's why a said, there might be a solution here. Maybe by defining something similar to a hidden label for every jump.

Well, it's easy to do with a macro. But for a standard process it'd be a bit of overkill, considering how barely noticeable the benefits are.

But, just for fun, I devised a simple modification that does an extreme version of this "overkill" and tracks additional hidden label for every line in source. This goes into PARSER.INC:
If you'd like to play with a modified version and determine how useful this can be, I encourage you to do so.