I found this project (flat assembler g) and was interested. I really like the macro syntax. But how do you structure the .inc files for the different architectures and how do you actually compile using them?

The fasm2 project is one example. Tersely, an architectural base acts as a preamble to development.

In the Linux environment, Jessé's work scaling fasmg features is another example.

Many different architectures pop up in Non-x86 and I'd follow their work.

Personally, I like the fasm2 strategy - even within a single architecture several execution environments can be tailored to.