flat assembler
Message board for the users of flat assembler.

Index > Compiler Internals > [fasmg] "preprocesses" double quotes to single quotes

Author
Thread Post new topic Reply to topic
Mike Gonta



Joined: 26 Dec 2010
Posts: 238
Location: the-ideom
Mike Gonta
Using the macro ? line& to process the individual lines of a file returns lines with matched double quotes converted to single quotes.
And for example: "xyz's" becomes 'xyz''s'.
I'm sure there is a good technical reason why this is done internally.
However, it would be nice (as a general purpose engine) to receive the original lines.

_________________
Mike Gonta
the-ideom - now you know how to compile

https://mikegonta.com
Post 06 May 2019, 00:03
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7496
Location: Kraków, Poland
Tomasz Grysztar
A tokenized line that fasm[g] operates on loses many properties of the original and it is generally not possible to recover the exact source text from it. For instance, if a source uses a C-like quote escaping scheme, you're going to lose it no matter what.

But when you need to work with the original sequence of characters, there are some tricks that make it possible, like in this example I prepared recently when talking about a similar problem with jacobly:
Code:
RAW_SOURCE = ''

macro READ_RAW_SOURCE path
        if RAW_SOURCE <> path
                RAW_SOURCE = path
                virtual at 0
                        file path
                        pos = 0
                        linepos = 0
                        linenum = 1
                        while pos <= $
                                if pos < $
                                        load c:byte from pos
                                else
                                        c = 10
                                end if
                                if c = 13 | c = 10
                                        repeat 1, n:linenum
                                                load RAW_SOURCE.n : pos-linepos from linepos
                                        end repeat
                                        linenum = linenum + 1
                                        if pos+1 < $
                                                load d:byte from pos+1
                                                if (c = 13 & d = 10) | (c = 10 & d = 13)
                                                        pos = pos + 1
                                                end if
                                        end if
                                        linepos = pos + 1
                                end if
                                pos = pos + 1
                        end while
                end virtual
        end if
end macro

macro ?! line&
        match =go =wild: foo, line
                READ_RAW_SOURCE __FILE__
                repeat 1, n:__LINE__
                        display RAW_SOURCE.n,13,10
                end repeat
        else
                display `line,13,10
        end match
end macro

        db ? ; comment
go wild:     this line's going to be read raw from the source; in entirety
go wild:     "\""    
Post 06 May 2019, 08:23
View user's profile Send private message Visit poster's website Reply with quote
Mike Gonta



Joined: 26 Dec 2010
Posts: 238
Location: the-ideom
Mike Gonta
Perfect.
I was wondering why the match doesn't work in this example.
Code:
macro ?! line&
  READ_RAW_SOURCE __FILE__
  repeat 1, n:__LINE__
    source = RAW_SOURCE.n
  end repeat
  match =go? foo , source
    display source,13,10
  else
;    display source,13,10
  end match
end macro

        db ? ; comment
go wild:     this line's going to be read raw from the source; in entirety
go wild:     "\""    

_________________
Mike Gonta
the-ideom - now you know how to compile

https://mikegonta.com
Post 06 May 2019, 20:40
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar
Assembly Artist


Joined: 16 Jun 2003
Posts: 7496
Location: Kraków, Poland
Tomasz Grysztar
Mike Gonta wrote:
I was wondering why the match doesn't work in this example.
The RAW_SOURCE.n is a string value. You can operate on it numerically, as on any other string:
Code:
  if source and 0FFFFh = 'go' | source and 0FFFFh = 'Go' | source and 0FFFFh = 'GO' | source and 0FFFFh = 'gO'
    display source,13,10
  end if    
MATCH, however, operates on tokenized text. It is possible to tokenize a string at assembly time with EVAL:
Code:
  repeat 1, n:__LINE__
    eval 'define source ',RAW_SOURCE.n
  end repeat
  match =go? foo , source
    display `foo,13,10
  end match    
But then you have gone the full circle, because tokenizing the raw text is getting you the same what you had in "line" argument already.

Therefore: in your example use "line" when you need to MATCH, use "source" when you need to work with original, raw text.
Post 07 May 2019, 07:37
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2019, Tomasz Grysztar.

Powered by rwasa.