[fasmg] Area Tools:

Index > Macroinstructions > [fasmg] Area Tools:

Author

Thread

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 30 Jun 2023, 09:28

Thought it might be useful to have a thread to hopefully consolidate area tools. Areas are a wonderful concept of the fasmg language. They can gather data progressively in parts, and/or be stored to files (making it easier to debug complex data processes or have multiple products from similar processing).

AreaContents: store area content byte string into a symbol - probably a good place to start.
xorout: encode output with XOR operation, using an area as a key.

Here is a byte frequency counting area tool:

Code:

match =FileName,FileName
        display 10,'usage: fasmg freqdump.g -i "FileName equ ''your_file''"',10
        err 'missing FileName expected'
else
        virtual at 0
                source::
                file FileName
        end virtual

        AreaFrequency source

        ; we can verify total equals length
        total = 0
        postpone
                assert total = sizeof source
        end postpone

        ; display array of values
        repeat 256, i:0
                repeat 1, N:source.i
                        display `N,','
                        total = total + N
dd N
                end repeat
        end repeat
end match

calminstruction AreaFrequency area*
        local offset, var, val, i
        compute offset, 0

        ; set all byte frequency counts to zero
        compute i, 256
clear:
        compute i, i - 1
        arrange var, area.i
        publish var, offset
        check i
        jyes clear
more:
        load val, area:offset, 1
        compute val, 0+val ; convert string to number
        arrange var, area.val
        compute val, var + 1
        publish var, val

        compute offset, offset + 1
        check offset < sizeof area
        jyes more
end calminstruction

... as more area tools develop, I will try to link them here.

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

Last edited by bitRAKE on 20 Dec 2024, 16:53; edited 1 time in total

30 Jun 2023, 09:28

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 01 Sep 2023, 02:06

Previously, I reformatted some JSON to be kind to editors. Now we do the reverse using some of the newest features of fasmg - pack JSON removing all whitespace:

Code:

match =FileName,FileName
        display 10,'usage: fasmg json.pack.g -i "FileName equ ''metaModel.json''"',10
        err 'missing FileName expected'
else
        virtual at 0
                source::
                file FileName
        end virtual
end match

calminstruction JSON_Pack area*
        local offset, outset, val
        compute outset, -1
        compute offset, -1
value:
        compute offset, offset + 1
        check offset = sizeof area
        jyes done
        load val, area:offset, 1
        check val = ' '
        jyes value
        check val = 9
        jyes value
        check val = 10
        jyes value
        check val = 13
        jyes value
        ; store verbatim
        compute outset, outset + 1
        store area:outset, 1, val
        check val = '"'
        jno value
quoted: ; assume valid JSON - all quotes close
        compute offset, offset + 1
        load val, area:offset, 1
        ; store verbatim
        compute outset, outset + 1
        store area:outset, 1, val
        check val = '"'
        jno quoted
        ; was it an escaped quote?
        load val, area:offset-1, 1
        check val = '\'
        jyes quoted
        jump value
done:
        ; define new length to area, SIZEOF no good!
        compute outset, outset + 1
        arrange val, area.=packed =:== outset
        assemble val
end calminstruction

format binary as 'json'
JSON_Pack source
load package:source.packed from source:0
db package

... there are many ways to code this, but the routine is part of a larger work and needed to be inplace updated. (Depending on your usecase, it's also useful to have a version of this routine that converts string escapes to their UTF-8 equivalents - simplifying down-stream processing.)

01 Sep 2023, 02:06

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 12 Nov 2024, 22:04

This area tool replaces an array of tags with their corresponding values:

Code:

virtual
TTT::
file __FILE__
end virtual

iterate <TAG,           VALUE>,\
        "{{VEGE}}",     "<<PEPPER>>",\
        "{{ANIMAL}}",   "<<BEAR>>",\
        "{{COMMAND}}",  "<<TESTING>>"

        define tags     TAG
        define values   VALUE
end iterate

TagReplaced TTT


calminstruction TagReplaced area*
        local offset,mark,bytes,tag,T,V
        compute offset, -1
new_range:
        compute mark, offset

tv:     take tags,T
        take values,V
        jyes tv

go:     compute offset, offset + 1
        check offset < sizeof area
        jno tail

        load tag, area:offset, 1
        check tag = '{'
        jno go

vt:     take T,tags
        take V,values
        jno tv

        check offset + lengthof T <= sizeof area
        jno vt

        load tag, area:offset, lengthof T
        check tag = T
        jno vt

        compute bytes, offset-mark-1
        check bytes > 0
        jno up
        load tag, area:1+mark, bytes
        emit bytes, tag ; output skipped bytes

up:     emit lengthof V, V
        compute offset, offset + lengthof T - 1
        jump new_range

tail:   compute bytes, offset-mark-1
        check bytes > 0
        jno done
        load tag, area:1+mark, bytes
        emit bytes, tag ; output skipped bytes
done:
end calminstruction

The real power is feeding a database into this macro to perform something analogous to a mail-merge if you're that old. Smile

Or just parameterized templating.

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

12 Nov 2024, 22:04

bitRAKE

Joined: 21 Jul 2003
Posts: 4073
Location: vpcmpistri

bitRAKE 24 Nov 2024, 11:18

The area tag replacement can be used in a multi-pass form, but it's a little tricky to get working. This advanced usage shows a sequence of areas being progressively refined until no replacements are possible. For example, this could be used to expand formal grammars.

An important note: although restartout cannot be used in a virtual, the final area is all that is stored to disk and redefining an output area is not an error. Of course, it's possible to save all intermediate steps as well:

Code:

virtual at 0 as 'result' bappend `%
        TagReplaced DRAFT.%,replacements
        load NOW:$-$$ from $$
end virtual

... making the output name unique within the repeat loop.

Description:		Download
Filename:	templatings.alm
Filesize:	1.68 KB
Downloaded:	52 Time(s)

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

24 Nov 2024, 11:18

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum