virtual at 0
unicode_data::
  file "UnicodeData-11.0.0d1.txt"
end virtual

struc column n
  local x, position, value
  x = 0
  position = 0
  repeat n
    while x <> ';'
      load x: byte from unicode_data: + line + position
      position = position + 1
    end while
    x = 0
  end repeat
  value = '0'
  while 1
    load x: byte from unicode_data: + line + position
    if x = ';'
      break
    end if
    position = position + 1
    value = (value shl 8) + x
  end while
  . = value
end struc

macro next_line
  local x
  x = 0
  while x <> 0x0A
    load x: byte from unicode_data: + line
    line = line + 1
  end while
end macro

struc hex2bin value
  local x
  result = 0
  repeat 5, n: 0
    result = result shl 4
    x = (value shr (32 - n * 8)) and 0xFF
    x = x - '0'
    if x > 9
      x = x - 7
    end if
    result = result + x
  end repeat
  . = result
end struc

  line = 0

;macro unicode_parser
;  local x, unicode
  repeat 10000
    x column 0
    next_line
    unicode hex2bin x
    dd unicode
  end repeat
;end macro

;  unicode_parser    

"hxmfv" is a number of version in the lowmem branch, which is a separately maintained fork at the moment. I have not merged this variant into the trunk, because in other cases (like fasmg self-hosting) it is slower. If this variant works better for you, please use checkouts from this branch instead of the official packages (which are made from the trunk).

Tomasz Grysztar wrote:

"hxmfv" is a number of version in the lowmem branch, which is a separately maintained fork at the moment. I have not merged this variant into the trunk, because in other cases (like fasmg self-hosting) it is slower. If this variant works better for you, please use checkouts from this branch instead of the official packages (which are made from the trunk).

All versions are the same time when not used as a macro. It is only the newer versions which have the large
difference between macro and non-macro when both methods are run on the same version.

The differences between the trunk and the "lowmem" branch are exactly in how the macros are handled. The newest version at the moment is actually the "hyhy9" in the "lowmem" branch, please try it. Because my guess it not related to which version is more recent, only whether it is from the trunk or the branch.

hyhy9 self hosted = 39.1
hyhy9 example inline = 11.0
hyhy9 example as macro = 11.4

hygtl self hosted = 37.8
hygtl example inline = 11.0
hygtl example as macro = 16.9    

hxmfv actual program inline = 63.2
hxmfv actual program as macro = 64.0

hyhy9 actual program inline = 68.5
hyhy9 actual program as macro = 95.3

hygtl actual program inline = 55.0
hygtl actual program as macro = 965.1    

However, my actual program which parses 16,584 lines of the text file and creates a rule based compressed table
takes 10X longer as a macro verses inline (15 times longer than hxmfv).

... a semicolon separated csv spreadsheet.

Please try the new version "hykpg", it fixes the actual underlying problem.