flat assembler
Message board for the users of flat assembler.

Index > Projects and Ideas > Principles for creating multi-developer programs with fasm

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20340
Location: In your JS exploiting you and your system
revolution 15 Feb 2008, 18:41
[This document is a work in progress and is intended to be periodically added to and updated]


Introduction:

This will provide a method for users of fasm to be able to use when collaborating on a software project with a specific goal in mind. This hopes to provide a basic framework, both in the coding sense, and the philosophical sense, that will ease the production of programs with more than one developer involved. It is hoped that projects like this (fasm browser) can benefit towards achieving a goal faster and easier by following basic guidelines outlined here.


Goals:

To provide a simple, yet broadly applicable, framework for developers to follow that allows code written by independent producers to seamlessly combine together within known parameters. Specific code constructs given in this document may not work if and when fasm is updated, but the hope is that the general concept is clear and changes can be minimal to keep the idea valid.


In Scope:

All significantly sized fasm based projects that involve multiple developers. That is, not a small piece of code that has been "tweaked" by many people along the way.

Providing a code library and general framework for program layout and testing.


Out of Scope:

This document does not proscribe methods of dividing the given project into tasks, nor does it proscribe any methods of inter-communication between developers. Basically, each project owner will still need to break the project into logical tasks and assign work based upon what is appropriate for the project and how that is done is not covered here.

This document is being primarily designed for non-commercial projects, so commercial project are considered out of scope. Anyone is free to use these guidelines for any purpose they wish, although it may not be fully appropriate in a commercial setting. Most notable is the lack of a stakeholder or customer, and only goes as far as specifying the project owner.


Assumptions:

At the time of writing the use of fasm 1.67 is implicit. Later versions may come along and naturally this can be updated from time to time to reflect any changes that may be necessary due to code incompatibilities and/or enhancements.


General principles/guidelines:

0. Project definition (it's scope and goals)
1. Minimum requirements
2. Allocation of labour
3. Calling conventions and naming standards
4. Divisibility of code
5. Common code base
6. Module management
7. Testing assurance
8. Debugging and conditional code
9. Documentation and commenting
10. Version numbering and regression

Appendices:

A. Macros for module disambiguation
B. Macros for global data definition
C. Macros for Win32 API interfacing
D. Example of a module file
E. Example of a program file

Principle 0: Project definition (it's scope and goals)

No multi-developer project can succeed unless the scope and goals are known. This is plainly evident by the fact that without a set of parameters to meet there is no way to judge the level of success. Therefore all such projects must outline in advance the scope and goals. The exact details of how to achieve the goals do not need to be decided until later. Also note that changing the scope and/or goals mid-project can be very damaging to the chances the project will succeed in achieving it's goals.

The project must have an owner, someone that is ultimately responsible for the project overview and delegation. This does not mean that the owner is to be answerable to all, it is just a way of providing a centralised (and hopefully, considered and reasonable) control mechanism.

Scope includes what the project is intended to cover and what it is not intended to cover. It is important to know just what is in-scope and out of scope. Code written to do things out of scope is not to be included for any reason.

The goals must be clear and preferably concise. This includes: 1) a specific end point, a target, 2) performance measures and indicators, 3) timelines, etc. Most importantly with goals, is they must be achievable with the given resources. It does no good to set very high goals that are just not possible to achieve.

Everybody involved must agree upon the goals, with no exception. Bad goals are worse than no goals at all.


Principle 1: Minimum requirements

Once the project has the scope and goals defined then decide how to best achieve them. There must be a standard minimum platform decided upon. This includes, but is not limited to, calling standard, memory availability, CPU architecture/speed, networking, storage requirements, OS, number of developers, etc. Some requirements will be self-evident based upon the project goals, but other requirements should be explicitly stated where appropriate.


Principle 2: Allocation of labour

Developers should register their willingness to work on certain parts of the project based upon their own skills, desires and/or expertise. In areas that are oversubscribed developers should be willing to accept other tasks that are within their skills but are lacking in coverage from others, that is, if a developer has a unique skill then they should be prepared to work in that area. The owner of the project should keep an up-to-date list of who is assigned which task(s) along with expected completion times.


Principle 3: Calling conventions and naming standards

Every piece of code must comply with the calling standard decided upon. If the project is cross-platform or based upon a particular OS then certain external calling standards will have already been imposed by the interface. However, the internal calling standard can still differ if it is deemed appropriate to achieve the goals.

Global variables and labels should be kept to an absolute minimum if the project uses multi-threading.

Individual names of function should ideally be prefixed with an all-uppercase common string that is unique to each module. Using the module filename (without the extension) name as a prefix is desirable where appropriate.

Some suggestions.

    -For fixed constants use all uppercase.
    -For global memory variable names use all uppercase with a leading underscore.
    -For local memory variable names use all lowercase with or without a leading underscore.
    -For function names use all lowercase with internal underscore spacing after the prefix
    -For function parameter names use all lowercase with internal underscore spacing.
    -For structure names use all uppercase.
    -For structure member names use all lowercase with internal underscore spacing.


Principle 4: Divisibility of code

Code is divided into programs and modules. A module (.inc) is used to provide some basic functional block and would usually be developed by one person that was assigned the task. A program (.asm) used to generate executable code or object files. Both module and program files may or may not require other external modules, but any external file it does directly require shall be "included" as needed. Both modules and program files may not "include" other modules that it does not directly access or require. Any indirect module requirements should be subsequently included by each of the child modules that require them.

Every code module must be able to be compiled as a "stand alone unit", but a module is not required to generate useful code when compiled alone. Where it says "not required to generate useful code" it means that it is not expected to generate code that can be run as an .exe, .dll or object file, but it may generate some binary data that would be useful code in a proper file format. A module will need an outer program file before it can generate directly useful code. If a module requires, and is dependent upon, other modules then it must "include" them as needed. No module shall be required to be included more than once. No module shall be allowed to be included more than once. Global data (both initialised and uninitialised) that is module specific shall also be defined directly within the module. This requires special management to achieve this in fasm by use of macros to automatically control the inclusion.

Example:

    Module A; depends upon module B & C; and defines global data D
    Module B; depends upon module A & C; and defines global data E
    Module C; depends upon module A & B; and defines global data F

    And we want to create an .exe file with the following layout:

    Section 0:
      Module A
      Module B
      Module C
    Section 1:
      Global data D
      Global data E
      Global data F

Normally in fasm this cannot be done if we are to achieve the "stand alone unit" requirement as stated above. So we need to use a method of dependency control. This is especially poignant with system libraries and common macro files. Appendix A shows the set of macros used to achieve the "stand alone unit" and "no multiple inclusion" requirements.


Principle 5: Common code base

The project owner is responsible for providing (or arranging the provision of) the common code base. That is, a base set of modules that are not dependant upon external modules. Internally these base module sets can be inter-dependant upon each other. This code base would most probably provide things like OS binding constants and labels, and general constants. It might also provide a set of base functions that are generic and generally useful for all modules.


Principle 6: Module management

Based upon the project nature and how it divides into specific areas the modules can be organised logically into sections. Depending upon the number of sections it may be suitable to split each broad area into separate folders/directories. The file system layout should be announced early in the project during the requirements generation. The file system layout can be expanded later as the need requires during project progression. A flat file system might be appropriate for some projects while a tree file system might be better for other projects. Due to specifics of how fasm operates it is advisable to make all filenames in lower case and also to enforce the use of relative pathnames at all times. The reason for lower case and relative paths is to allow for macros to properly disambiguate situations where multiple modules will all include the same libraries. It also helps to provide some degree of independence upon exact locality on the host file system as each user may have a different base folder they use to assemble the project.

Only one file associated with an EXE or DLL (or similar OS dependant executable/linkable file) shall have an extension ".asm". All module files shall have the extension ".inc". This makes it obvious which files are intended for which purpose. Modules by their very nature are not designed to be stand alone programs, thus the .inc extension serves to make it clear it is to be "included" within another file. DLL and EXE (.o files etc.) files will each have a corresponding .asm file that is assembled to create it.

When the target file is not directly executable, but is intended to be linked with other object files (as when mixing with C/C++/HLL) it shall still have a .asm extension to signify that it is the file to assemble to create the object file.

Unless absolutely required, no .inc or .asm file shall be dependant upon the fasm switch -d. Files that are dependant upon using the -d command line parameter must clearly and explicitly state near the beginning of the file why it is used and exactly what can be defined and it's effect upon the output file. It shall also provide a mechanism for a reasonable default set of values if the -d switch is not used.

The project owner shall collect all module and program files and collate them into a common resource, either on demand, as and when required, or at regular intervals, for all participants to download and/or reference.


Principle 7: Testing assurance

Every module file shall have an associated test generator file. The test generator shall be intended to set up a basic minimal environment that is required to exercise all of modules functions. The test file should also provide some means of assurance that the module is performing as desired and required. The test generator file will have a .asm extension and should be run-able on the host system of the developer. For projects with mixed platform developers, the test file should ideally aim to use something like a virtual machine to emulate the host system of the majority of the developers, but this is not a requirement.


Principle 8: Debugging and conditional code

TBD


Principle 9: Documentation and commenting

Arguably the most hated part of software development. For the purposes here, this will define only a suggested minimum requirement for documentation. After all, it would not be good to discourage participation in a project based upon onerous documentation requirements. Individual projects may choose to make higher stipulations for more extensive documentation.

At a minimum each program and module shall briefly state at, or near, the top of the file what the program/module is intended to do. It shall then list all of the callable functions, with parameter names. After that one can optionally list each function again with an expanded description of it's intended behaviour, I/O parameter ranges and any special situations. Extra comments above this minimum are encouraged. The overall project shall have a text "readme" file that has all of the decided goals, scope, file structure, requirements etc.

Inside a program or module file it is not generally useful to describe each and every line of code by stating it's direct function. The operation of a line of code is self-evident from the mnemonic codes, it does not need to be described again. Example:

    mov eax,3 ;move three into register eax

In the above, based upon the x86 CPU documentation, the code "mov eax,3" is clear and concise about it's operation, the comment "move three into register eax" is unnecessary repetition. A more useful comment would be this:

    mov eax,3 ;a pixel has 3 colours, red, green and blue, one byte per colour

In the above, it describes what the value three is actually intended to be. With this type of comment we can reasonably expect the value of eax to be used inside some type of graphical code, and we might also expect that it could be a multiplier for something later. The right type of comment should give information to help in understanding what the overall code is doing or intended to do.


Principle 10. Version numbering and regression

Each file shall have a version number as assigned by the developer. Numbers can be assigned arbitrarily but must be in a strictly increasing order. The project specification may want to impose tighter control over the numbering with the project owner controlling the major version number and allow minor numbering to be decided by each individual developer. The project owner shall keep all previous versions of each of the files in case of a regression requirement. These "old versions" of files do not need to be included into the standard resource that is separately maintained. Any suitable method of naming files can be used but generally simply appending the old version numbers to the file name (after the normal extension) would be adequate.


Appendix A: Macros for module disambiguation

Below: The includeonce macro, the heart of the multiple inclusion paradigm towards "stand alone" modules each including their own dependencies.
Code:
;file name: includeonce.inc v0.0
;this file can itself be included multiple times
;allows for multiple inclusion of files with only one instance being instantiated

;usage: in the program/module file use "include 'includeonce.inc'", then
;for each required dependency use "includeonce 'myfile.inc'"

;there is an optional feature where arbitrary code can be generated after the file name
;i.e. You can use "includeonce 'myfile.inc', <display 'myfile.inc included',13,10>"

;ideally all files name should be in lowercase and all paths should be relative

match =files@included,files@included{
        macro nest_includeonce\{
              macro includeonce path,[instr]\\{\\common
                  file@include equ path
                       match head path tail,files@included\\\{file@include equ\\\}
                 match head path,files@included\\\{file@include equ\\\}
                      match file,file@include\\\{
                         files@included equ files@included path
                              nest_includeonce
                            include file
                                purge includeonce
                           irp i,instr\\\\{i\\\\}
                    \\\}
                \\}
  \}nest_includeonce
    files@included equ x
}    


Appendix B: Macros for global data definition

Below: The global data definition macros.
Code:
;file name: globals.inc v0.0
;
;sets up five macros to define global data:
;
;.idata           - declaration of initialised global data
;.udata             - declaration of uninitialised global data
;.sdata           - declaration of initialised global string data
;.bcode              - declaration of one-time start up executed code
;.ecode             - declaration of one-time shut down executed code
;
;and sets up another 5 macros to instantiate the global data
;
;iData           - instantiate the initialised global data
;uData             - instantiate the uninitialised global data
;sData           - instantiate the initialised global string data
;bCode              - instantiate the one-time start up executed code
;eCode             - instantiate the one-time shut down executed code


;All macros are used like this:
;
;
;        .idata
;     {
;             global_var1 rd 1
;   }
;
;
;   .bcode
;     {
;             call MODULE_init_lut
;               ... other arbitrary code
;   }
;
;
;   iData
;      bCode


;initialised global data

.idata_list equ

macro .idata {
 local z
     .idata_list equ .idata_list,z
       macro z
}

macro iData {
 match data_list,.idata_list \{
                irp instr,data_list \\{
                      instr
               \\}
  \}
}


;uninitialised global data

.udata_list equ

macro .udata {
  local z
     .udata_list equ .udata_list,z
       macro z
}

macro uData {
 match data_list,.udata_list \{
                irp instr,data_list \\{
                      instr
               \\}
  \}
}


;global string data

.sdata_list equ

macro .sdata {
 local z
     .sdata_list equ .sdata_list,z
       macro z
}

macro sData {
 match data_list,.sdata_list \{
                irp instr,data_list \\{
                      instr
               \\}
  \}
}


;one time start-up initialisation/allocation code

.bcode_list equ

macro .bcode {
   local z
     .bcode_list equ .bcode_list,z
       macro z
}

macro bCode {
 match code_list,.bcode_list \{
                irp instr,code_list \\{
                      instr
               \\}
  \}
}


;one time shut-down deinitialisation/deallocation code

.ecode_list equ

macro .ecode {
      local z
     .ecode_list equ .ecode_list,z
       macro z
}

macro eCode {
 match code_list,.ecode_list \{
                irp instr,code_list \\{
                      instr
               \\}
  \}
}    


Appendix C: Macros for Win32 API interfacing

In order to support the principle of "stand alone" modules, each module must include all of it's dependencies. These macros provide a mechanism for each module to include the Win32 API definitions at any time.
Code:
;file name: 'import32.inc' v0.0
;
;provides 5 macros for Windows 32bit interfaces
;
;libraryonce TagLabel, SystemFilename, APIDefinitionFile, [..., ..., ...]
;put_library_table
;put_import_lookups
;put_library_names
;put_import_names


;e.g. To use the user32.dll API:
;
;    libraryonce user32,'USER32.DLL','api\user32.inc'


;In your .asm project file, use like this:
;
;       include 'includeonce.inc'
;        includeonce 'import32.inc'
;
;  libraryonce     user32,'USER32.DLL','api\user32.inc',\
;                       gdi32,'GDI32.DLL','api\gdi32.inc'
;
;        ... your code ...
;
;     align 4
;    data import
;                put_library_table
;          put_import_lookups
;         put_library_names
;          put_import_names
;   end data
;
;
;Note: if you also need the defined equates and structures included with
;the fasm windows package you will need to include those separately. e.g.
;
;       includeonce 'win32a.inc'
; includeonce 'macro\struct.inc'
;  includeonce 'equates\user32.inc'
;
;In your overall project these may be wrapped into a common code base file
;
;Or you can make a wrapper for each API. e.g. A file named 'win32user.inc'
;
;   include 'includeonce.inc'
;        includeonce 'import32.inc'
;       includeonce 'win32a.inc'
; includeonce 'macro\struct.inc'
;  includeonce 'equates\user32.inc'
;        libraryonce user32,'USER32.DLL','api\user32.inc'
;

include 'includeonce.inc'

.library_tag_list equ

macro libraryonce [dll_tag,dll_name,import_file] {
    forward
    libraryonce@tag equ dll_tag
 match head dll_tag tail,.library_tag_list\{libraryonce@tag equ\}
        match head dll_tag,.library_tag_list\{libraryonce@tag equ\}
     match any,libraryonce@tag\{
           .library_tag_list equ .library_tag_list,dll_tag
             dll_tag#.module_name equ dll_name
           match any,import_file\\{includeonce import_file\\}
    \}
}


macro put_library_table {
        match =,tag_list,.library_tag_list \{
         if $ and 3
                  rb -1 ;align me, I need alignment provided externally
               end if
              irp tag,tag_list \\{
                 if defined tag\\#.required & tag\\#.required
                                dd RVA tag\\#.lookup,0,0,RVA tag\\#.string,RVA tag\\#.lookup
                  end if
              \\}
          dd 0,0,0,0,0
        \}
}


macro put_library_names {
        match =,tag_list,.library_tag_list \{
         irp tag,tag_list \\{
                 if defined tag\\#.required & tag\\#.required
                                rb (-RVA $) and 3
                           tag\\#.string db tag\\#.module_name,0
                   end if
              \\}
  \}
}


macro put_import_names {}
macro put_import_lookups {rb (-RVA $) and 3}


macro import name,[label,string] {
    common
        macro put_import_names \{
             put_import_names
    forward
             if used label
                       rb RVA $ and 1
                      name#.#label: db 0,0,string,0
           end if
    common
        \}
    macro put_import_lookups \{
           put_import_lookups
          name#.lookup:
    forward
            if used label
                       label dd RVA name#.#label
           end if
    common
                if $-name#.lookup
                   dd 0
                        name#.required=1
            end if
      \}
}    

Code:
;file name: 'win32a.inc' v0.0
;ASCII default Win32 interfacing
;
;provides 3 macros for Windows 32bit interfaces
;
;api name, ...
;pushd value
;.end EntryAddress
;
;provides 1 structure for Windows 32bit interfaces
;
;TCHAR


;api name, ...
;
;The macro 'api' is an ASCIIfier, that is, it allows calls the generic Win32
;API functions without reference to the final 'A' or 'W'. In this file it
;defines all relevant functions as 'A' version.


;pushd value
;
;Provides functionality for pushing various alternative types of parameters.
;
;e.g.
;
; pushd 'string',13,10,0
;   pushd 'string'
;           -pushes the address of the text onto the stack
;             -and declares the string in the the .sdata section
;
;    pushd double ptr variable
;  pushd double [variable]
;    pushd double variable
;              -pushes the value of two consecutive dwords onto the stack
;
;    pushd addr expression
;              -pushes the effective address of the expression onto the stack


;.end EntryAddress
;
;finalises the code section with .bcode and .ecode instructions
;creates the data section
;places the .idata and .sdata tables
;defines the .udata tables
;creates the import section
;places the import tables


;In your .asm project file, use like this:
;
;include 'includeonce.inc'
;includeonce 'win32a.inc'
;
;MyEntry:
;... your code ...
;      ret
;
;.end MyEntry
;

include 'includeonce.inc'
includeonce 'globals.inc'
includeonce 'import32.inc'

struc TCHAR [val] {
    common
      match any,val \{
              . db val
    \}
    match ,val \{
         . db ?
      \}
}

macro api [name] {
    forward
    if used name
                label name dword at name#A
  end if
}

macro pushd [value] {
    common
    match first=,more,value \{
            \local ..string
            pushd ..string
              .sdata
              \\{
                  if used ..string
                            ..string: db value,0
                    end if
              \\}
          pushd equ
   \}
    match pushd =addr var,pushd value \{
          \local ..opcode,..address
          virtual at 0
                        label ..address at var
                      mov eax,dword[..address]
                    load ..opcode from 0
                end virtual
         if ..opcode = 0A1h
                  pushd var
           else
                        lea edx,[..address]
                 push edx
            end if
              pushd equ
   \}
    match pushd =double [var],pushd value \{
              push dword[var+4]
           push dword[var]
             pushd equ
   \}
    match pushd =double =ptr var,pushd value \{
           push dword[var+4]
           push dword[var]
             pushd equ
   \}
    match pushd =double num,pushd value \{
                \local ..high,..low
                virtual at 0
                        dq num
                      load ..low dword from 0
                     load ..high dword from 4
            end virtual
         pushd ..high
                pushd ..low
         pushd equ
   \}
    match pushd,pushd \{
          \local ..string
            if value eqtype ''
                        pushd ..string
                      .sdata
                      \\{
                          if used ..string
                                    ..string: db value,0
                            end if
                      \\}
          else
                        pushd value
         end if
              pushd equ
   \}
    restore pushd
}

libraryonce kernel32,    'KERNEL32.DLL',       'api\kernel32.inc'

macro .end entry_address {

    entry   $
   bCode
       call    entry_address
       push    eax
 eCode
       call    [ExitProcess]

   section '.data' data readable writeable

       iData
       sData
       rb (-RVA $) and 3
   virtual
             uData
               ..udata_size=$-$$
   end virtual
 rb ..udata_size

 section '.idata' data import readable writeable

       put_library_table
   put_import_lookups
  put_library_names
   put_import_names
}

virtual
   xchg eax,eax
        ..NotYetFormatted=($-$$-1)
end virtual
if ..NotYetFormatted
   format PE GUI 4.0
end if

section '.code' code readable executable    

Code:
;file name: 'win32w.inc' v0.0
;UNICODE default Win32 interfacing
;
;provides 3 macros for Windows 32bit interfaces
;
;api name, ...
;pushd value
;.end EntryAddress
;
;provides 1 structure for Windows 32bit interfaces
;
;TCHAR


;api name, ...
;
;The macro 'api' is an UNICODEifier, that is, it allows calls the generic Win32
;API functions without reference to the final 'A' or 'W'. In this file it
;defines all relevant functions as 'W' version.


;pushd value
;
;Provides functionality for pushing various alternative types of parameters.
;
;e.g.
;
; pushd 'string',13,10,0
;   pushd 'string'
;           -pushes the address of the text onto the stack
;             -and declares the string in the the .sdata section
;
;    pushd double ptr variable
;  pushd double [variable]
;    pushd double variable
;              -pushes the value of two consecutive dwords onto the stack
;
;    pushd addr expression
;              -pushes the effective address of the expression onto the stack


;.end EntryAddress
;
;finalises the code section with .bcode and .ecode instructions
;creates the data section
;places the .idata and .sdata tables
;defines the .udata tables
;creates the import section
;places the import tables


;In your .asm project file, use like this:
;
;include 'includeonce.inc'
;includeonce 'win32w.inc'
;
;MyEntry:
;... your code ...
;      ret
;
;.end MyEntry
;

include 'includeonce.inc'
includeonce 'globals.inc'
includeonce 'import32.inc'

struc TCHAR [val] {
    common
      match any,val \{
              . du val
    \}
    match ,val \{
         . du ?
      \}
}

macro api [name] {
    forward
    if used name
                label name dword at name#W
  end if
}

macro pushd [value] {
    common
    match first=,more,value \{
            \local ..string
            pushd ..string
              .sdata
              \\{
                  if used ..string
                            rb (-RVA $) and 1
                           ..string: du value,0
                    end if
              \\}
          pushd equ
   \}
    match pushd =addr var,pushd value \{
          \local ..opcode,..address
          virtual at 0
                        label ..address at var
                      mov eax,dword[..address]
                    load ..opcode from 0
                end virtual
         if ..opcode = 0A1h
                  pushd var
           else
                        lea edx,[..address]
                 push edx
            end if
              pushd equ
   \}
    match pushd =double [var],pushd value \{
              push dword[var+4]
           push dword[var]
             pushd equ
   \}
    match pushd =double =ptr var,pushd value \{
           push dword[var+4]
           push dword[var]
             pushd equ
   \}
    match pushd =double num,pushd value \{
                \local ..high,..low
                virtual at 0
                        dq num
                      load ..low dword from 0
                     load ..high dword from 4
            end virtual
         pushd ..high
                pushd ..low
         pushd equ
   \}
    match pushd,pushd \{
          \local ..string
            if value eqtype ''
                        pushd ..string
                      .sdata
                      \\{
                          if used ..string
                                    rb (-RVA $) and 1
                                   ..string: du value,0
                            end if
                      \\}
          else
                        pushd value
         end if
              pushd equ
   \}
    restore pushd
}

libraryonce kernel32,    'KERNEL32.DLL',       'api\kernel32.inc'

macro .end entry_address {

    entry   $
   bCode
       call    entry_address
       push    eax
 eCode
       call    [ExitProcess]

   section '.data' data readable writeable

       iData
       sData
       rb (-RVA $) and 3
   virtual
             uData
               ..udata_size=$-$$
   end virtual
 rb ..udata_size

 section '.idata' data import readable writeable

       put_library_table
   put_import_lookups
  put_library_names
   put_import_names
}

virtual
   xchg eax,eax
        ..NotYetFormatted=($-$$-1)
end virtual
if ..NotYetFormatted
   format PE GUI 4.0
end if

section '.code' code readable executable    


Appendix D: Example of a module file

Code:
TBD    


Appendix E: Example of a program file

Code:
TBD    


V6, Copyright (C) 2008, revolution


Last edited by revolution on 17 Feb 2008, 23:00; edited 6 times in total
Post 15 Feb 2008, 18:41
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20340
Location: In your JS exploiting you and your system
revolution 16 Feb 2008, 18:30
Now up to the fifth draft, still some way to go.
Post 16 Feb 2008, 18:30
View user's profile Send private message Visit poster's website Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 12802
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 0010456
sleepsleep 17 Feb 2008, 17:50
good, mind if i copy and print it out and distribute to others?

i will not forget to print this one too
Quote:
V5, Copyright (C) 2008, revolution
Post 17 Feb 2008, 17:50
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20340
Location: In your JS exploiting you and your system
revolution 17 Feb 2008, 19:43
sleepsleep: It's not finished yet so you might like to wait before printing. But if you still want to you are welcome to print it at any time.
Post 17 Feb 2008, 19:43
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4335
Location: Now
edfed 17 Feb 2008, 22:43
do you need drawings? to illustrate some ideas?
as we try to interpret html, we'll try to make the html file as an help file directlly implemented in the source, to display it in case of connection problem.
this html file can be any tutorial or course, in txt , asm, or htm format. with or without images...
Post 17 Feb 2008, 22:43
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20340
Location: In your JS exploiting you and your system
revolution 17 Feb 2008, 23:03
edfed: What type of drawings do you have in mind? I am willing to consider if they are good quality and not subject to copyright restrictions.
Post 17 Feb 2008, 23:03
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 18 Feb 2008, 21:56
Sorry, i just quickly browsed through yet. Here's my few practical objections:

Point 4 looks very strongly tight to using FASM's abilities to compile directly to executable. Very purpose of object/library files are to prevent doing that in big projects. Direct executable output is nice for small project, but a really bad idea for big project (especially if heavily macroed). Instead, you should adopt "proper" way, eg. objects and makefiles. This also allows you to use libraries whose source are not interfaced according to your rules (eg. all code existing to this day).

Point 8: You will get some debug info (names of public symbols) with using objects. This can be done without objects too, but requires again more code.

Point 10 will make project maintaining much harder and redundant, if some version control system (SVN, CVS, git, whatever) is used. I think SVN is pretty enough for this.

Appendix B: "start-up autoexecute code" is infortunate Pascalish idea. Adding one call instruction to startup code ONCE PER PROJECT is not worth of some automatization. Worse problem is, you will often find that you need to reinitialize many modules... i smell some code duplication...
Post 18 Feb 2008, 21:56
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4335
Location: Now
edfed 18 Feb 2008, 22:19
just drawings, with pen, pencils and chinese-ink, and about copyrights, you bad consider me!
if i proposed to illustrate this doc, it's free and totally copy left

about the pictures, you ask, i draw and scan, you see and emit critics and i correct...
Post 18 Feb 2008, 22:19
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20340
Location: In your JS exploiting you and your system
revolution 19 Feb 2008, 04:07
Thanks vid,

Your comment on point 4 is solvable, I will update this again soon and show how it can accommodate your concerns there.

Your comment on point 10 is correct. I have made it too specific. I will rewrite this to make it more friendly for alternative control setups.

Your comment on Appendix B is curious. The intention was that if a module has a requirement for code to run at start-up (say some lut initialisation, or maybe a mutex) then there is no need to contact the author of another file to include your start-up line. This is also intended to allow modules to be included into other code without having to add lines to external files to make it work.
Post 19 Feb 2008, 04:07
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 19 Feb 2008, 21:35
revolution: as for appendix B, at time when you write your module, you don't know how would it be used, and whether it would be enough to initialize once per program run, or many times.

also, "use code without doing anything with it/understanding it" is utopia, that would never work with anything bit more complicated than template. Anyone who uses some code module should study it's usage, and calling one initialization procedure is easiest that there can be.
Post 19 Feb 2008, 21:35
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4335
Location: Now
edfed 19 Feb 2008, 22:06
sometimes, there is no need of init, in case of external context for exemple..
but, for system functions, init parts are really needed...
Post 19 Feb 2008, 22:06
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20340
Location: In your JS exploiting you and your system
revolution 20 Feb 2008, 01:59
vid: Do you have an example of what you mean about it never working? I have used this method for some time now and had nothing but success.
Post 20 Feb 2008, 01:59
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 20 Feb 2008, 02:33
well, imagine you are doing some (in-)game graphics engine. You naturally put initialization into this init part. You make prototype game, engine is initalized at the beginning, all works nice. And then you decide you want to add menu. Going to menu will require to uninitialize ingame graphics engine, and use menu graphics engine (pretty common, i hope you have seen this case). Then when you want to return to game, you must reinitialize game graphics engine - but whoops, that's not possible.

Simply: you shouldn't enforce HOW is the module used (always initialized at startup). Just provide some init call, some uninit call, and let one who uses module decide how/when/how many times is he gonna use it. Calling ONCE PER PROJECT is really not worth of doing any such extra features, and there is big risk people will use it in wrong manner (eg. for initialization of modules which aren't 100% granted to need to be initialized at startup).

If you use this auto-initi feature, you force user of your module to always initialize it at startup. That is completely unnescessary limitation.

PS: you would also need to solve proper order of initialization based on inter-module dependency, and that's NOT gonna be easy!
Post 20 Feb 2008, 02:33
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20340
Location: In your JS exploiting you and your system
revolution 20 Feb 2008, 02:57
The init code is not intended for things that can be called again at runtime, that would just be silly. I mentioned an LUT or MUTEX, they are good examples where calling once at startup is all that is needed, they never get re-initialised. So in your example, where something needs to be initialised/deinitialised many times during a session, using the .bcode/.ecode macro is not appropriate.

Where you mention about adding a menu, this is where the design goals have changed. I already mentioned that changing the goals can be damaging to the success of the project and your example shows a case why this is so damaging. People write code based upon the initial set of goals and it works for those goals. When later the goals change then your code breaks and suddenly the project fails. Principle 0, very important!

I give you my example. In my Rijndael code I setup the LUT at startup. After that, no matter how many threads are running, they can all read the table when needed. Since the table is fixed and unchanging there is no need for each thread to call the init function.

BTW: There is no compulsion to use it. Wink
Post 20 Feb 2008, 02:57
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 20 Feb 2008, 11:50
Quote:
The init code is not intended for things that can be called again at runtime, that would just be silly. I mentioned an LUT or MUTEX, they are good examples where calling once at startup is all that is needed, they never get re-initialised. So in your example, where something needs to be initialised/deinitialised many times during a session, using the .bcode/.ecode macro is not appropriate.


few things:
1. you never know how at time of writing whether it's gonna be used once or not
2. such data which really always need to be initialized only once are quite rare. i quess if you add this part, 90% times it would be used for general initialization of module, not just for part of initialization which never needs to uninitialized / reinitialized
3. some modules might be optional, and for not wasting resources, you may not want to initialize it at all if it's disabled, not even it's mutexes etc.
4. if only initialize-once part of initialization will be in this init section, then caller would have to call proper initialization anyway. What's the point in this then?
5. Mutex is thing which should be reinitialized every time module is reinitialized. not a good example.

seriously, this is

also few problems with this:
- proper order of initializing that i mentioned, based on module dependency
- if you automatize this, it may be impossible to use circular module dependency
Post 20 Feb 2008, 11:50
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20340
Location: In your JS exploiting you and your system
revolution 20 Feb 2008, 12:03
But it is not intended for many of the things you mention so don't use it there, easy fix. I intend it for situations where order of initialisation is not an issue and where de/re-initialisation is not going to happen. Like any tool, it can be used, or it can be abused. If it is not abused then it is not a problem. And I agree, it can be misused, so treat with due consideration.
Post 20 Feb 2008, 12:03
View user's profile Send private message Visit poster's website Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 20 Feb 2008, 12:13
my point is that this "tool" is almost unworthy (saves few lines per project at best), but it is prompting anyone who don't properly understand it to misuse it (for real initialization). There is big risk and little-to-no gain from this feature.
Post 20 Feb 2008, 12:13
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
vid
Verbosity in development


Joined: 05 Sep 2003
Posts: 7105
Location: Slovakia
vid 20 Feb 2008, 12:18
by the way, one practical advice for development of this: start small. don't plan big system from beginning, that way you're gonna screw it. Start with usual bunch of files without any extra features, and slowly add new features. Otherwise you will have to redesign it few times (unless you're some ultra-genius or have lot of experience in designing such frameworks)
Post 20 Feb 2008, 12:18
View user's profile Send private message Visit poster's website AIM Address MSN Messenger ICQ Number Reply with quote
booter



Joined: 08 Dec 2006
Posts: 67
booter 28 Dec 2009, 08:58
@revolution
Is this development on hold?
Anyway, I have several objections:
- I don't really like the idea of "minimal requirements" as a starting point. This approach usually leads to massive updates to provide functionality that was not thought at the beginning. I would say, prioritized "maximum requirements" would work better. For ex. instead of "works in Win98" it should be "any version of windows, Win98 is priority"
- Naming standards are way too subjective. People may have their own preferences. I believe the naming conventions should be up to "project owner".
- The requirement that Every code module must be able to be compiled as a "stand alone unit" is very stong and it is not really necessary if we consider that every module has to have its own test program, which is where it has to be compiled (principle 7).
- DateTime stamp already uniquely identifies the module version, but "real" versioning should be done on Project level (for ex. with SVN).
- Unfortunaly, Fasm does not have support for modularity, particularly there is no simple way to avoid name conflicts.
Regarding the aggregation of data, initialization, and termination from all includes I have the solution that is not as heavy as yours, though it has its own limitations. I will post details on my framework in a separate topic.

Main Program to compile
Code:
entry Start
;*******************************************************
include '%fasminc%\win32ax.inc'
include 'MyMacro.ASM'
;*******************************************************
section '.code' code readable executable
;-- -----------------------------------------------------
include 'A.ASM'
include 'B.ASM'
include 'C.ASM'
;-------------------------------------------------------
Start:
  Init ; initialization code from all includes
..................
Finish:
  Term ; termination code from all includes
  callW32  ExitProcess,0
;-------------------------------------------------------
section '.data' data readable writeable
  Data ; data from all includes
  X  dd 0
..................
    

MyMacro.inc
Code:
macro Init { } ; to collect initialization code from includes
macro Term { } ; to collect termination code from includes
macro Data { } ; to collect data from includes
.................
    

In modules
Code:
................
macro Init { Init
; module initialization code
................
}
macro Term { Term
; module termination code
................
}
macro Data { Data
; module data
................
}
    


@vid
I completely disagree with "Direct executable output is nice for small project, but a really bad idea for big project", and especially "adopt "proper" way, eg. objects and makefiles". Makefiles and headers are the ugliest "features" of C/C++. I can hardly imagin a project that may frustrate developers with whole recompilation each time they need to test their updstes. Changes in big projects require more time to test/debug, so compilation takes relatively insignificant time regardless of the project size. If you really want independent modularity, create your DLLs. All this obj-crap came from very old times when computers were etremely slow.
Post 28 Dec 2009, 08:58
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20340
Location: In your JS exploiting you and your system
revolution 28 Dec 2009, 09:28
Thanks for your recommendations. I make some comment as below:
booter wrote:
Is this development on hold?
No. Just "slow" at the moment.
booter wrote:
Anyway, I have several objections:
- I don't really like the idea of "minimal requirements" as a starting point. This approach usually leads to massive updates to provide functionality that was not thought at the beginning. I would say, prioritized "maximum requirements" would work better. For ex. instead of "works in Win98" it should be "any version of windows, Win98 is priority"
I think that wishy-washy goals (weasel words) are not a good idea. Giving priority to one OS means what for the other OSes? That they can be ignored, or that they must be done but not yet? You can make it a goal without the weasel words. "First sub-goal: Must work in Win98. Once achieved then then next staging post: Must work in Win2K." and so on. Those are fixed and clear goals that are achieved as the project progresses.
booter wrote:
- Naming standards are way too subjective. People may have their own preferences. I believe the naming conventions should be up to "project owner".
Yes. I had only made suggestions, you are free to choose a different style. The important point is it must be clear and unambiguous to all participants.
booter wrote:
- The requirement that Every code module must be able to be compiled as a "stand alone unit" is very strong and it is not really necessary if we consider that every module has to have its own test program, which is where it has to be compiled (principle 7).
This is necessary if you want to reuse modules on other projects. I should have explained that but I forgot. The idea is that you can assure yourself that the module does not need anything external from some other file included by the test program. It is actually not difficult to achieve if you follow the other principles.
booter wrote:
- DateTime stamp already uniquely identifies the module version, but "real" versioning should be done on Project level (for ex. with SVN).
Sure, a good idea. If you have SVN available.
booter wrote:
- Unfortunaly, Fasm does not have support for modularity, particularly there is no simple way to avoid name conflicts.
Regarding the aggregation of data, initialization, and termination from all includes I have the solution that is not as heavy as yours, though it has its own limitations. I will post details on my framework in a separate topic.

Main Program to compile
Code:
entry Start
;*******************************************************
include '%fasminc%\win32ax.inc'
include 'MyMacro.ASM'
;*******************************************************
section '.code' code readable executable
;-------------------------------------------------------
include 'A.ASM'
include 'B.ASM'
include 'C.ASM'
;-------------------------------------------------------
Start:
  Init ; initialization code from all includes
..................
Finish:
  Term ; termination code from all includes
  callW32  ExitProcess,0
;-------------------------------------------------------
section '.data' data readable writeable
  Data ; data from all includes
  X  dd 0
..................
    

MyMacro.inc
Code:
macro Init { } ; to collect initialization code from includes
macro Term { } ; to collect termination code from includes
macro Data { } ; to collect data from includes
.................
    

In modules
Code:
................
macro Init { Init
; module initialization code
................
}
macro Term { Term
; module termination code
................
}
macro Data { Data
; module data
................
}
    
Thanks for the suggestion. If those macros makes more sense to you then it should be okay also.
Post 28 Dec 2009, 09:28
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.