[fasmg] small error in line continuation with comments

Index > Compiler Internals > [fasmg] small error in line continuation with comments

Goto page 1, 2 Next

Author

Thread

bitRAKE

Joined: 21 Jul 2003
Posts: 4081
Location: vpcmpistri

bitRAKE 04 Dec 2018, 00:44

Just expanding the example from the manual and adding a comment to the end of line caused an error:

Code:

        iterate <name,value>, \; comment
          a,1, \
          b,2, \
          c,3
                name = value
        end iterate

Seems like a weird corner case, but I really do like to split these lines and comment. Very Happy

Using version g.ibh5n which is the most recent I see. Or, maybe this is no longer possible?

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

04 Dec 2018, 00:44

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 04 Dec 2018, 07:54

I broke it when I implemented RETAINCOMMENTS and ISOLATELINES. Thank you for letting me know, it should be fixed now (g.idtrm).

04 Dec 2018, 07:54

bitRAKE

Joined: 21 Jul 2003
Posts: 4081
Location: vpcmpistri

bitRAKE 10 Dec 2018, 07:05

Would you mind if I completely rewrote map.inc? Or should a just post small changes, like line 161, should remove implied lock on xchg:

Code:

        mov     edx,[ebx+Map.linked_blocks]
        mov     [ebx+Map.linked_blocks],eax
        mov     [eax],edx

...and I could refactor destroy_string_map to:

Code:

destroy_string_map:
; in: ebx - map
; preserves: esi, edi

@@:     xchg eax,ebx
        mov ebx,[eax] ; Map.linked_blocks must be first element
        call mfree
        test ebx,ebx
        jnz @B
        retn

Code:

hash_string:
; in: rsi - string, rcx = string length, zero for ASCIIZ string
; out: rcx = string length, edx = 32-bit hash
; preserves: ebx, esi, edi
        mov edx,FNV_OFFSET
        push rsi
        jrcxz hash_asciiz
        push rcx
    hash_known_length:
        lodsb
        xor dl,al
        imul edx,FNV_PRIME
        loop hash_known_length
        pop rcx
        pop rsi
        retn

    hash_asciiz:
        lodsb
        xor dl,al
        imul edx,FNV_PRIME
        inc ecx
        test al,al
        jnz hash_asciiz
    hash_ready:
        pop rsi
        retn

...etc.

10 Dec 2018, 07:05

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 10 Dec 2018, 10:20

I don't mind improvements to this code, especially considering that MAP.INC was the very first set of routines I created when started implementing fasmg, because I needed it for source cache in the beginning. For this purpose the performance was not important and the code was left mostly unchanged, according to the rule of "not fixing what is not broken". Though after all I ended up using it in a few more places (in addition to original source cache), so perhaps it could be worth improving... Did you notice any gain in regular fasmg use with your corrections?

bitRAKE wrote:

Code:

        mov ebx,[eax] ; Map.linked_blocks must be first element

With the style conventions I use in fasmg sources this should look like:

Code:

        assert  Map.linked_blocks = 0
        mov     ebx,[eax]       ; [eax+Map.linked_blocks] in the first run

10 Dec 2018, 10:20

bitRAKE

Joined: 21 Jul 2003
Posts: 4081
Location: vpcmpistri

bitRAKE 10 Dec 2018, 15:24

I wasn't really thinking about performance. It was more like "low hanging fruit". Seemed rather contained with little chance of me breaking something else. Hopefully, I will get more comfortable with the source and the style.

With regard to XCHG, modern processors will only lock the cache if the data is in the cache. So, I doubt it will have much impact in the present use.

Something else I noticed: hash_string includes the terminating zero in the hash of a string only when the string length is not provided. Assuming the zero isn't counted in the length. Therefor a use case exists where the same string could return different hashes. Obviously, this doesn't happen in FASMG - otherwise it wouldn't work. Does the design require the inclusion of the terminating null?

(So, far I think that it does. Appears to be only two uses which pass ECX: use_source counts the null, format_binary is a little harder to understand. All the others use the asciiz branch and hash the null.)

Maybe, I'm wrong, but I think hashing the null on strings weakens the low order bits of the hash - which is relied on by the map. Seems easy enough to skip it - I'm tracing down the length returned to see if that will create a problem.

10 Dec 2018, 15:24

bitRAKE

Joined: 21 Jul 2003
Posts: 4081
Location: vpcmpistri

bitRAKE 10 Dec 2018, 22:48

Rather than pondering here further, I tried some things.

My discovery is that strings aren't always strings. Yet, if I keep the null in the length everything appears to work okay. So, hashing without the null is fine because non-strings are always processed with their length. The converse is not true - some strings are not null terminating - which is strange.

Since it's working then I'm assuming there is another class of strings which are always called with their length, and they would never match regular null strings. File extension is one case, and number strings another.

10 Dec 2018, 22:48

bitRAKE

Joined: 21 Jul 2003
Posts: 4081
Location: vpcmpistri

bitRAKE 11 Dec 2018, 05:01

So far I've been able to slow it down by 5% with very small changes. Razz

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

11 Dec 2018, 05:01

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 11 Dec 2018, 06:43

Yes, in general any given map either always uses explicit length [byte strings] or always NULL-terminated ones. Because the two kinds are never mixed within the same map, this did not manifest as a bug.

The terminating zero needs to be counted into length, because when such string needs to be copied elsewhere (to a string storage), the terminating zero needs to be copied as well.

11 Dec 2018, 06:43

bitRAKE

Joined: 21 Jul 2003
Posts: 4081
Location: vpcmpistri

bitRAKE 20 Dec 2018, 23:26

manual.txt : 698,47 whey -> when

On a side note, something that always trips me up using fasmg is how ":", LABEL, and NAMESPACE do different but related things. I would think just because the name doesn't start with a "." that the base namespace changed, but it doesn't work that way.

Code:

BASE:
.x = 1 ; this is BASE.x

BASE.y:
.x = 2 ; this is also BASE.x?

To be fair, there is a lot in the manual on this, and I've read it a dozen times. I just lack an intuitive feel for what is happening.

manual.txt, pg 154 wrote:

When an identifier starts with a dot (in other words: when the name of the parent symbol is empty), it refers to the symbol in the namespace of the latest label that was defined in the current base namespace.

NAMESPACE actually would create a 'BASE.y' base, but neither ":" or label will create "BASE.y" as a parent - they just create the child ".y".

I've even tried "BASE.y.:", but fasmg will silently ignore the trailing "."

_________________
¯\(°_o)/¯ “languages are not safe - uses can be” Bjarne Stroustrup

20 Dec 2018, 23:26

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 21 Dec 2018, 07:19

bitRAKE wrote:

On a side note, something that always trips me up using fasmg is how ":", LABEL, and NAMESPACE do different but related things. I would think just because the name doesn't start with a "." that the base namespace changed, but it doesn't work that way.
Code:
BASE:
.x = 1 ; this is BASE.x

BASE.y:
.x = 2 ; this is also BASE.x?    

When designing the language I decided to define this mechanism using as simple set of rules as possible, but this simplicity does in fact lead to a quirk you mention.

That simple rule is that what the initial dot does is append to the previous label defined within current namespace. The quirk here is that unless you use NAMESPACE directive, current namespace stays the same. Current namespace is root for the entire length of your snippet. And the only label defined in that namespace is "BASE", so it is the "previous" label for the entire snippet, too.

The problem here is that "BASE.y" does not define a label in current namespace. It defines a label in "BASE" namespace. To make the initial dot "see" it, you would have to switch current namespace to "BASE":

Code:

BASE:
.x = 1 ; this is BASE.x

BASE.y:
namespace BASE
    .x = 2 ; this is BASE.y.x
end namespace
.x = 3 ; this is BASE.x again

bitRAKE wrote:

I've even tried "BASE.y.:", but fasmg will silently ignore the trailing "."

The trailing dot does make a very subtle difference that is completely irrelevant here. It is better to leave it to discuss another time.

21 Dec 2018, 07:19

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20461
Location: In your JS exploiting you and your system

revolution 21 Dec 2018, 08:04

So what is the result from:

Code:

BASE:
.x = 1 ;BASE.x = 1
ROOT.y:
.x = 1 ; ???

So labels with an internal dot do not change the current namespace?

21 Dec 2018, 08:04

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 21 Dec 2018, 08:18

revolution wrote:

So labels with an internal dot do not change the current namespace?

Labels never change the current namespace, it can only be changed with NAMESPACE directive. Label can only change what the "previous" label within the current namespace is. And when label contains dot internally, it is not a label in current namespace (it becomes a "previous" label in some other one).

21 Dec 2018, 08:18

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20461
Location: In your JS exploiting you and your system

revolution 21 Dec 2018, 08:35

So:

Code:

BASE: ;namespace = BASE
ROOT: ;namespace still BASE?
.x=1; BASE.x = 1?

21 Dec 2018, 08:35

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 21 Dec 2018, 08:48

No.

Code:

; we start off in some namespace, perhaps the main root (but the source could be encapsulated)
BASE: ; namespace still the same, last label in this namespace is now BASE
ROOT: ; namespace still the same, last label in this namespace is now ROOT
.x=1  ; ROOT.x = 1, last label in ROOT namespace is now x, but last label in the outer one is still ROOT

21 Dec 2018, 08:48

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20461
Location: In your JS exploiting you and your system

revolution 21 Dec 2018, 08:50

So why not:

Code:

BASE: ; namespace still the same, last label in this namespace is now BASE
ROOT.y: ; namespace still the same, last label in this namespace is now ROOT.y
.x=1 ; ROOT.y.x = 1

21 Dec 2018, 08:50

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 21 Dec 2018, 08:52

revolution wrote:

So why not:

Code:

BASE: ; namespace still the same, last label in this namespace is now BASE
ROOT.y: ; namespace still the same, last label in this namespace is now ROOT.y
.x=1 ; ROOT.y.x = 1

Because "ROOT.y" defines a label (named "y") in "ROOT" namespace, not in the outer one. "ROOT.y" is not a name of the symbol, it is a "path" to symbol, consisting of two names. Think of it like directories and files.

Last edited by Tomasz Grysztar on 21 Dec 2018, 08:54; edited 1 time in total

21 Dec 2018, 08:52

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20461
Location: In your JS exploiting you and your system

revolution 21 Dec 2018, 08:54

Okay. And

Code:

BASE: ; namespace still the same, last label in this namespace is now BASE
ROOT.a.b.c: ; namespace still the same, last label in this namespace is now ROOT?
.x=1 ; ROOT.x = 1

21 Dec 2018, 08:54

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 21 Dec 2018, 08:55

revolution wrote:

Okay. And

Code:

BASE: ; namespace still the same, last label in this namespace is now BASE
ROOT.a.b.c: ; namespace still the same, last label in this namespace is now ROOT?
.x=1 ; ROOT.x = 1

No, because we did not define "ROOT" label in the outer namespace. The last label is still "BASE". This only defined "c" label in the namespace that resides at "ROOT.a.b" path, it did not define any label in any other namespace.

21 Dec 2018, 08:55

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20461
Location: In your JS exploiting you and your system

revolution 21 Dec 2018, 09:17

Okay. So ROOT, ROOT.a and ROOT.a.b are not labels, they are only namespace paths.

And ROOT.a.b.c is not really a label, it is a namespace path ROOT.a.b with the .c label inside it?

21 Dec 2018, 09:17

Tomasz Grysztar

Joined: 16 Jun 2003
Posts: 8359
Location: Kraków, Poland

Tomasz Grysztar 21 Dec 2018, 09:58

revolution wrote:

Okay. So ROOT, ROOT.a and ROOT.a.b are not labels, they are only namespace paths.

And ROOT.a.b.c is not really a label, it is a namespace path ROOT.a.b with the .c label inside it?

Yes, exactly.

21 Dec 2018, 09:58

Goto page 1, 2 Next

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum