flat assembler
Message board for the users of flat assembler.

Index > Programming Language Design > On my new assembler

Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9, 10  Next
Author
Thread Post new topic Reply to topic
jacobly



Joined: 04 Feb 2016
Posts: 44
jacobly 27 Jul 2017, 22:31
Another unintuitive thing I ran into was that
Code:
rb 1 - 2 bad_token 3    
errors with "value out of allowed range" but not "extra characters on line" (even after increasing the error limit) which would have helped me figure out the actual issue much faster. Note that
Code:
rb 2 - 1 bad_token    
does indeed error with "extra characters on line". In hindsight, the out of range error makes sense, but I'm not sure that it should suppress the extra characters error.

Edit: Also, a few minor syntactical features from other compilers and assemblers that would be nice, but not strictly necessary and aren't really worth writing a macro for:
binary literals: %1, 0b1
octal literals: @7, 0o7 (or 0c7 because o looks like 0)
\ followed by anything but whitespace and a comment acts like a newline (this is often used in z80 code for things like rrca \ rrca \ rrca \ rrca).


Last edited by jacobly on 27 Jul 2017, 22:59; edited 1 time in total
Post 27 Jul 2017, 22:31
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 28 Jul 2017, 07:40
jacobly wrote:
Another unintuitive thing I ran into was that
Code:
rb 1 - 2 bad_token 3    
errors with "value out of allowed range" but not "extra characters on line" (even after increasing the error limit) which would have helped me figure out the actual issue much faster. Note that
Code:
rb 2 - 1 bad_token    
does indeed error with "extra characters on line". In hindsight, the out of range error makes sense, but I'm not sure that it should suppress the extra characters error.
I agree, the second error should not have been suppressed here. This is something that is easy to fix in this specific case, but may still happen sometimes in other places, as the implementation of every directive decides on its own when to abort and stop registering more errors.

jacobly wrote:
Also, a few minor syntactical features from other compilers and assemblers that would be nice, but not strictly necessary and aren't really worth writing a macro for:
binary literals: %1, 0b1
octal literals: @7, 0o7 (or 0c7 because o looks like 0)
It is generally a good thing to recognize more number formats, though it reduces assembler's ability to detect typos (because the family of texts considered "correct" becomes larger). But doing that for classes of symbols that do not start with digits has another problem - they would no longer be allowed as names then, and both "%1" and "@1" may be needed as names in some situations. For example "%1" and similar are used as names in NASM-like macro syntax if one wanted to emulate it, and "@1" may be used in some local label schemes.

jacobly wrote:
\ followed by anything but whitespace and a comment acts like a newline (this is often used in z80 code for things like rrca \ rrca \ rrca \ rrca).
This is something that can be relatively easily implemented by macro, while making it an internal feature would again limit the abilities of redefining syntax. It is currently possible to MATCH the "\" characters inside the line and process them accordingly to the desired syntax rules, if it was treated by preprocessor before passing lines to assembly it would not be feasible. For example fasm 1 used backslash in a different meaning in its macro syntax, and if one wanted to emulate some of the fasm 1 syntax with fasmg macros, having backslashes passed as a part of macro arguments would be important.

There is no perfect solution to these problems, it is all about compromise. For example, fasmg still implements line concatenation and comments in a traditional way (compatible with fasm 1), even though this, for example, makes it impossible to use the ";" character in a custom syntax. I tried to find a balance in considering what might be more usually expected from fasm.
Post 28 Jul 2017, 07:40
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 28 Jul 2017, 08:20
Version hvj4j notices the "extra character on line" in the case you reported.
Post 28 Jul 2017, 08:20
View user's profile Send private message Visit poster's website Reply with quote
jacobly



Joined: 04 Feb 2016
Posts: 44
jacobly 28 Jul 2017, 11:45
Hmm, I did not actually realize that those were used for other purposes by other assemblers (the only local label schemes I've ever seen are +_, @f, 0f), so that is indeed a good reason to not support them. I've also just come to the realization that by the time you are trying to support another assembler's syntax, you pretty much need a macro ? anyway, so there's no real reason to add things to fasmg that can already be supported in that way.

Is there any reason that a postpone ? block would be executed before all other values are resolved? Maybe I'm misunderstanding its purpose, but I'm trying to avoid a very slow loop that is the result of an unresolved variable evaluating to 0, but the loop really only needs to be executed once after the last pass.
Post 28 Jul 2017, 11:45
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 28 Jul 2017, 12:46
jacobly wrote:
Is there any reason that a postpone ? block would be executed before all other values are resolved? Maybe I'm misunderstanding its purpose, but I'm trying to avoid a very slow loop that is the result of an unresolved variable evaluating to 0, but the loop really only needs to be executed once after the last pass.
The "postpone ?" needs not to define any constant or variables that may affect the other parts of source, because this is going to mess up the resolving and the additional passes are going to be needed.
Post 28 Jul 2017, 12:46
View user's profile Send private message Visit poster's website Reply with quote
ProMiNick



Joined: 24 Mar 2012
Posts: 798
Location: Russian Federation, Sochi
ProMiNick 28 Jul 2017, 13:36
is for fasmg can be added version log?: what is done (like it done in fasm1).
Post 28 Jul 2017, 13:36
View user's profile Send private message Send e-mail Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 28 Jul 2017, 14:30
ProMiNick wrote:
is for fasmg can be added version log?: what is done (like it done in fasm1).
fasmg has something even better, that is the repository log.
Post 28 Jul 2017, 14:30
View user's profile Send private message Visit poster's website Reply with quote
ProMiNick



Joined: 24 Mar 2012
Posts: 798
Location: Russian Federation, Sochi
ProMiNick 28 Jul 2017, 14:44
And where it is? Link? (Or I `m stupid or my english is bad:I don`t found it) I feel 2nd...
Post 28 Jul 2017, 14:44
View user's profile Send private message Send e-mail Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 28 Jul 2017, 15:19
I have linked it in a few places in this very thread.
Post 28 Jul 2017, 15:19
View user's profile Send private message Visit poster's website Reply with quote
jacobly



Joined: 04 Feb 2016
Posts: 44
jacobly 28 Jul 2017, 17:08
Tomasz Grysztar wrote:
The "postpone ?" needs not to define any constant or variables that may affect the other parts of source, because this is going to mess up the resolving and the additional passes are going to be needed.
Even when I delete every symbol definition in the postpone, it still gets runs early. According to my debug fasmg, it always gets run during pass 1, but then gets correctly deferred until the last pass.

Code:
a = b
b = 1 shl 24
postpone ?
    repeat b - a
    end repeat
end postpone    
This takes > 1 second to compile, yet changing it to repeat a - b compiles instantly because then the first pass sees a negative repeat count and skips it.

Edit: In my more complex program, it actually gets run during passes 2 and 3, yet if I artificially add passes, everything works fine. Maybe I just have to actually use the variables that are defined solely for the postpone ? outside it for them to resolve first?


Last edited by jacobly on 28 Jul 2017, 17:16; edited 1 time in total
Post 28 Jul 2017, 17:08
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 28 Jul 2017, 17:16
Ah, I see, you have found a kind of architectural bug there.
Post 28 Jul 2017, 17:16
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 28 Jul 2017, 17:28
Please try it with the "hvjms" version.
Post 28 Jul 2017, 17:28
View user's profile Send private message Visit poster's website Reply with quote
jacobly



Joined: 04 Feb 2016
Posts: 44
jacobly 28 Jul 2017, 17:44
Tomasz Grysztar wrote:
Please try it with the "hvjms" version.
Thanks, it works perfectly!
Post 28 Jul 2017, 17:44
View user's profile Send private message Reply with quote
jacobly



Joined: 04 Feb 2016
Posts: 44
jacobly 30 Jul 2017, 05:37
Is it intentional for macro ? to get the lines of an included file but for macro ?! to skip them? I can sort of get around this with an interesting trick, but I still miss any unconditional macros at the top of the included file:
Code:
macro ? temp&
    purge ?
    macro ?! line&
        display `line, 10
        line
    end macro
    temp
end macro
include 'test.inc'
purge ?    


I also just realized that trying to use macro ?! to parse syntax like .endif isn't going to work anyway since conditionally excluded lines aren't sent to macro ?!, so I will have to try something else.
Post 30 Jul 2017, 05:37
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 30 Jul 2017, 09:36
jacobly wrote:
Is it intentional for macro ? to get the lines of an included file but for macro ?! to skip them?
The "?!" macro intercepts every line with no exception, therefore the line that contains INCLUDE directive gets replaced, too. Even if you end up including the file (by assembling the original line inside the interceptor) you do so inside the macro, so it no longer applies.

Also, since an unconditional interceptor catches all the lines that follow, no matter what they contain, it is useful when you need to completely alter the syntax in a section of source text and interpret all the lines "from scratch". It is not a good tool for selective preprocessing - to correctly assemble things like control directives and macro definitions you'd need to collect entire blocks of lines and then execute them appropriately.

jacobly wrote:
I can sort of get around this with an interesting trick, but I still miss any unconditional macros at the top of the included file (...)
This is a nice trick I didn't know, even though it's not perfect. But unfortunately, because of the way unconditional interception works, it is not possible to define such interceptor before the INCLUDE and have it applied to the lines inside the included file, as it ends up just eating the line containing INCLUDE statement. The best I can offer would be to use EVAL to insert the interceptor into an included text:
Code:
macro ?! line&

        match =INCLUDE? path, line
                local text
                virtual
                        db 'macro ?! l&',10
                        db '  display `l, 10',10
                        db '  l',10
                        db 'end macro',10
                        file path
                        db 10,'purge ?'
                        load text:$-$$ from $$
                end virtual
                eval text
        else match =PURGE names, line
                purge names
        else match any, line
                display 'Ignored: ',`any,13,10
        end match

end macro

include 'test.inc'    


jacobly wrote:
I also just realized that trying to use macro ?! to parse syntax like .endif isn't going to work anyway since conditionally excluded lines aren't sent to macro ?!, so I will have to try something else.
As noted above, all lines are processed by such macro, but keep in mind that macros are really just a text replacement, and if you replace one text with another inside a skipped block, these lines are still going to be skipped - unless the replacement creates a line like "end if" (like in the example in the manual).
Post 30 Jul 2017, 09:36
View user's profile Send private message Visit poster's website Reply with quote
jacobly



Joined: 04 Feb 2016
Posts: 44
jacobly 30 Jul 2017, 19:49
The eval trick doesn't actually work in my case because I'm using a local namespace so once you enter the include file, there's currently no way to even define an unconditional macro without preexisting context which eval would destroy. However, since I need neither label sizes nor dot-prefix style local labels, and since I will probably need to customize label defines anyway, I decided that it is easier to just redirect "label:" to "label := $" so that macros starting with a dot work anywhere inside the included file.
Post 30 Jul 2017, 19:49
View user's profile Send private message Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 30 Jul 2017, 20:47
jacobly wrote:
The eval trick doesn't actually work in my case because I'm using a local namespace so once you enter the include file, there's currently no way to even define an unconditional macro without preexisting context which eval would destroy.
If you EVAL entire file at once there is no difference in how it is processed compared to when INCLUDE processes it. The source text by default does not have any recognition context attached to it, only the symbolic values like parameters have.

jacobly wrote:
However, since I need neither label sizes nor dot-prefix style local labels, and since I will probably need to customize label defines anyway, I decided that it is easier to just redirect "label:" to "label := $" so that macros starting with a dot work anywhere inside the included file.
This sounds like a good idea. It is best done with "struc ?", as the interceptor is then only run when a line does in fact contain a label.
Post 30 Jul 2017, 20:47
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 04 Aug 2017, 18:14
Tomasz Grysztar wrote:
jacobly wrote:
On an unrelated note, I was having problems understanding and debugging multiple pass issues, so I ended up creating a debug version of fasmg that outputs the display buffer every pass (usually with a small pass limit < 10 or so). This was so invaluable that I was thinking maybe it could be added as a command line argument.
It would be an interesting option to have. Perhaps I could add it as another level to "-v" switch ("-v2"?).
I have added a hidden "-v2" option but I made it work a bit differently - I remembered that I once used a tweaked fasmg version that showed DISPLAY messages in real time to see the results of a script that was slowly searching for solutions to a problem. This similarly has an effect that you get messages from all the consecutive passes, but it also allows to trace the assembly even within a single pass that takes a long time.

Additionally I experimented a bit with potential further levels of verbosity, like showing all the preprocessed lines in real time, but when testing it I noticed that it simply overwhelms with mostly useless information. A well-placed use of DISPLAY with the "-v2" option should be a better tool to debug macros rather than to blindly display every preprocessed line.
Post 04 Aug 2017, 18:14
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 11 Aug 2017, 16:20
The "-v2" switch turned out invaluable when I needed to figure out what in my MachO macros was causing few of my snippets to take excess passes. Thank you, jacobly, for this suggestion!
Post 11 Aug 2017, 16:20
View user's profile Send private message Visit poster's website Reply with quote
Tomasz Grysztar



Joined: 16 Jun 2003
Posts: 8351
Location: Kraków, Poland
Tomasz Grysztar 18 Sep 2017, 17:24
I have added another tiny bit of compatibility with fasm 1: fasmg now allows FORMAT BINARY AS 'EXT' directive so that it is possible to specify a file extension. Therefore the output path may now be omitted from the command line and when it is not present, fasmg generates the name of the output file automatically from the name of the source, just like fasm 1.

There are also some differences - the default extension is the lack of one. So the following command is not going to create any file:
Code:
fasmg nul -iDisplay(120)    

These additions open up a way for the option of generating multiple files from one source, which I've been planning for years. It would be limited to declaring the extensions of the produced files, analogously to the main file.
Post 18 Sep 2017, 17:24
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9, 10  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.