flat assembler
Message board for the users of flat assembler.

Index > Main > F-TNT project begin discussion.

Goto page 1, 2, 3, 4  Next
Author
Thread Post new topic Reply to topic
JohnFound



Joined: 16 Jun 2003
Posts: 3502
Location: Bulgaria
JohnFound
As a result of some PM negotiations Wink between Pelaillo, me and decard, I decided to begin this thread about the project (temporary name) F-TNT (fasm tips and tricks) system.

The main idea is described here: http://board.flatassembler.net/topic.php?t=2760

So, my opinion is that the main format of the stored articles, have to be more simple than html or xml. Maybe plain text or some very simple markup for simple formating: for example _underlined text_ and *bold text* combined with FASM syntax highlighting on the client side (on server side for online browsing)

This approach will give us following advantages:

1. small size of the database
2. easy implementation of the offline client browser not depending on external HTML
3. easy implementation of the offline search engine - without parsing HTML or XML
4. one common look&feel of all articles - of course this can be done with using XML but it is hard for parsing/rendering.
5. higher security - no script engines - no problems.

Regards.
Post 20 Dec 2004, 17:14
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
decard



Joined: 11 Sep 2003
Posts: 1092
Location: Poland
decard
I have some general ideas: let's discuss them Wink

1. name:
there are already some suggestions for the name:

F-TNT (fasm tips and trics) - current name... well I just don't like it
FADE (free assembly documentation effort) - I would change free assembly to flat assembler Very Happy

my suggestion
FARC (flat assembler resource center) - similar to above, but has more "extended" meaning. Actually FA can be also Free Assembly, I don't know which is better.


2. the database
- main database stored as a MySQL database
- it is accessible from both client application and website
- some special PHP file will be used by client app for downloading articles

(client will send simple HTTP requests to obtain proper article; PHP will handle MySQL access)

3. storing data
- articles stored as plain text with some simple formatting; it can be a small set of HTML tags: <h1>, <b>, <i>, <code>, maybe a few more. The tags will be displayed directly by the web browser, client can parse or just ignore them. Line breaks stored as \n, not as <br> (to saves space).

- structure describing an article (draft):
- article ID
- content
- authors
- modification date/time (so that client can download latest versions)

- keyword list: currently I have no complete idea how to implement them Sad It can be be done in two ways:
a) list of keywords, each one has its index; every article has attached an array of keyword indexes
b) each keyword has an array of article indexes
Good keyword system is important: it would help searching artiles.

4. Table of contents.
Everything could be simply sorted by operating systems, but for example some snippets are os-independent... my suggestion is that each article would have some variables (category,OS,...) assigned that will define it's type to allow some filtering, so for example user can request "string functions working under Windows".

5. Contributing articles:
three alternatives:
a) everyone can submit their modifications (may be dangerous, but easiest)
b) everyone can submit modifications, when it happens moderators will recive email with new article so they can verify it (mails send with PHP). This option is my favorite.
c) authors would sent their modifications to moderators, they will check the
articles and publish them if they are good. It could be partially automated.
(safe, but a lot of borring work for moderators)
Post 20 Dec 2004, 20:06
View user's profile Send private message Visit poster's website Reply with quote
vbVeryBeginner



Joined: 15 Aug 2004
Posts: 884
Location: \\world\asia\malaysia
vbVeryBeginner
hi, johnfound
here is a bit of my idea, i don't know if it is differ on architecture with urs or not, so i share it so you could have more choices Smile

1. we build a win32 .exe as a client offline viewer with sqlite database or mysql or etc... (named it as u wish)
2. we create some php scripts to handle our above client on some existing phpbb test board where the script would function as below

2a. retrieve PLAIN text article based on article id
2b. retrieve a list of submitted article from a specific time
2c. retrive title only instead of whole article (like newgroup viewer) so that users could check which one they wanna download.
2d.

3. user should REGISTER their offline viewer with board before they are allowed to use the script, so we give the user an offline viewer (OV) serial. or ID or etc...

4. since the OV is coming with a sqlite or ... database, so we just use their technology to search, insert update delete and ...

on the submitting article or ..., i think it is nice if we could have a viewer first then later move on to the submitting part, one reason is, there are a lot of information already on board, whether on fasm or win32 asm board, and if we could grab those information, well that is good coz lot of answer actually inside them.

one problem is, off course, unless they willing to share their database info with our OV. instead of making it specific to fasm, i guess we would have more users if we target it for all available board.

lastly, i don't know if i present my mind nicely or ...

actually, the mission is to convert all board to newsgroup based type method.

sincerely
sulaiman chang
Post 20 Dec 2004, 21:16
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3502
Location: Bulgaria
JohnFound
decard wrote:
1. name:


Well, any of them is good. Maybe later we will choose one. For now my opinion is to choose one "internal" or "work" name simply to call the project. What about "the project Thingamy". Wink

Quote:
2. the database
- main database stored as a MySQL database
- it is accessible from both client application and website
- some special PHP file will be used by client app for downloading articles
(client will send simple HTTP requests to obtain proper article; PHP will handle MySQL access)


Fully agree, except the database. IMHO, SQLite is better:
1. For now it is the only variant for the offline assemly written client - because of its small size.
2. It will allow equal structure of the client and server databases - SQLite is accessible with Linux and Windows version.

Quote:

3. storing data
- articles stored as plain text with some simple formatting; it can be a small set of HTML tags: <h1>, <b>, <i>, <code>, maybe a few more. The tags will be displayed directly by the web browser, client can parse or just ignore them. Line breaks stored as \n, not as <br> (to saves space).


So, \n will be ignored by the normal browser and the text will become "sticked". Also, IMHO it will be good if the text of the articles is directly compilable by FASM - this will make the things easy.
For example the plain descriptive text can be marked with some tag at the begining of the row (';T' for example) and this kind of text can contains any tags. The browser (offline) will simply strip the comments and will format the line depending of the tags (btw: we can try to make this behaviour even for AsmEdit syntax highlighter)
NB: The prototype D-TNT uses simple plain text editor with Pascal syntax highligher - IMHO it is the Delphi control "SynEdit"

Quote:
- keyword list: currently I have no complete idea how to implement them Sad


Maybe we have to leave this to the database engine to index the fields and simply to use SQL queries for searching?

Quote:
4. Table of contents.


IMHO, there should be central managed structure of categories. This structure will allow additions/editions only by administrators, not by the usual contributors.

Quote:
5. Contributing articles:


IMHO, it can be made simple:
1. Everyone can post new articles.
2. Articles can be deleted from the server only by moderators.
3. Articles can be deleted by offline database by any user (and marked, or not with flag "don't download again")
4. New articles can be posted only by offline client - this will limit the automatic spam of multiply posts.

This looks dangerous for me, but it works for Delphi people, why not to work for us as well?

Regards.
Post 20 Dec 2004, 21:32
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3502
Location: Bulgaria
JohnFound
vbVeryBeginner wrote:
hi, johnfound


Hi, sulaiman!

You simply type faster than me. Very Happy

I dissagree only with several things: The whole idea is not about discussions. It is not a message board. The main idea is to make information service that to have "auto-creation" feature.

I am against registration of users or authours. This will limit the userbase and authors.

About the FASM or general assembly, IMHO we have to make it for FASM. At least the sources will be open and everyone can install the same system on any other server. BTW: the client application can support list of servers so it will work for every database.

Regards
Post 20 Dec 2004, 21:40
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
decard



Joined: 11 Sep 2003
Posts: 1092
Location: Poland
decard
JohnFound wrote:
Fully agree, except the database. IMHO, SQLite is better:
1. For now it is the only variant for the offline assemly written client - because of its small size.
2. It will allow equal structure of the client and server databases - SQLite is accessible with Linux and Windows version.

But I mean the "global" database that is stored on server - here I have only a choice between MySQL and the DB based on textfiles. MySQL is obvious choice here. Of course client can store it in SQLite - we only have to design compatible tables that can be used in both database types. There's no need for advanced MySQL features here, so it won't be any problem.


Quote:

So, \n will be ignored by the normal browser and the text will become "sticked".

PHP script can change \n to <br> when rendering an article. But right, this additional bytes aren't really important Smile More important is not to make user type "<br>" on end of each line...

Quote:

Also, IMHO it will be good if the text of the articles is directly compilable by FASM - this will make the things easy.
For example the plain descriptive text can be marked with some tag at the begining of the row (';T' for example) and this kind of text can contains any tags. The browser (offline) will simply strip the comments and will format the line depending of the tags (btw: we can try to make this behaviour even for AsmEdit syntax highlighter)

IMHO it is unnecessary. And what if an article contains a just a few snippets (I think most articles will do)? I think it is enough to let user copy source and paste it in Fresh or FASMW...

Quote:

Maybe we have to leave this to the database engine to index the fields and simply to use SQL queries for searching?

I'm afraid it will be too slow if database is really big.

Quote:

IMHO, it can be made simple:
1. Everyone can post new articles.
2. Articles can be deleted from the server only by moderators.
3. Articles can be deleted by offline database by any user (and marked, or not with flag "don't download again")
4. New articles can be posted only by offline client - this will limit the automatic spam of multiply posts.

OK. We could allow posting articles from the website until client is ready.

regards
Post 20 Dec 2004, 22:48
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3502
Location: Bulgaria
JohnFound
decard wrote:
But I mean the "global" database that is stored on server - here I have only a choice between MySQL and the DB based on textfiles.

AFAIK, SQLite doen't need any install on the server. You simply put the file in the cgi-bin directory (of course your host have to allow cgi-bin).
We have to check SQLite docs for this...

Quote:
More important is not to make user type "<br>" on end of each line...

Fully agree. The most important of the format elements can be auto detected even from plain text: paragrafs, links, headings.

Quote:
I'm afraid it will be too slow if database is really big.

Well, we can't find out without some tests...
Post 20 Dec 2004, 23:08
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
roticv



Joined: 19 Jun 2003
Posts: 374
Location: Singapore
roticv
Hello guys,

Have not visited this forum for some time and there seem to be alot of changes/ Actually what you proposed JohnFound is pretty much like the language that is used in phpwiki...

Why can't we just stick to something we know very well like mysql? Very Happy

Regards,
Victor
Post 21 Dec 2004, 17:19
View user's profile Send private message Visit poster's website MSN Messenger Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3502
Location: Bulgaria
JohnFound
roticv wrote:
Actually what you proposed JohnFound is pretty much like the language that is used in phpwiki...


Yes, that is right. More or less different, but this is the idea. This format is easy for human reading and easy for machine parsing. It lacks a lot of features, but this is IMHO advantage not disadvantage. Wink

Quote:
Why can't we just stick to something we know very well like mysql?


Of course we can, but the main problem is that we can't use MySQL for the offline client, so we will be forced to use another DBase, that we know not so well.

Regards
Post 21 Dec 2004, 17:27
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3502
Location: Bulgaria
JohnFound
OK, I made some (little) work on the subject. You can download it here: FTnT.zip
It is the client side part, with database, that (for now) only reads the categories in the treeview control.
As far as I haven't ready category structure, I filled the database with files from one of my CD's, so here we have about 3000 nodes in the category tree and SQLite process them pretty quick.

btw: the program is small and IMHO is good example how to work with SQLite database from Fresh/FASM.

You can find the project of the database structure in the file "HelpSystem.txt" - any ideas are welcome.

Also, SQLite have one big dissadvantage, that the API convention is CDECL. I wonder, whether it is possible to be recompiled to use STDCALL? Please someone with C skils to give opinion.

Regards.

P.S. I included in the zip archive one console tool for SQLite database access - sqlite3.exe. It is not neccessary for the program, but it is usefull to execute queryes in the database.
Post 31 Dec 2004, 08:29
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
decard



Joined: 11 Sep 2003
Posts: 1092
Location: Poland
decard
and here's my "little" work on the subject Wink
It is the website. Actually now you can browse categories... and nothing else Very Happy Interesting that I came to similar idea about storing categories as John - the tables are almost identical. For the site design, I'm basing on Pelaillo's FADE project (it looks similar to PHP site, and it is really nice). Check it yourself:
http://fade.decard.net/showcat.php

I also thought about client - http://fade.decard.net/client/getcat.php can be used to upload category structure. I will output "raw" data, with \n between rows and spaces between fields. The format is really close to John's table:

create table categories (
id int not NULL primary key auto_increment,
parent int not NULL,
name varchar(128) not NULL,
description text not NULL
);

PHP source: http://decard.net/fade/fade.zip
Post 01 Jan 2005, 08:51
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3502
Location: Bulgaria
JohnFound
Hi, Decard.
Happy new year! Smile

You make very good work with server side.

Only one request for getcat.php. In order to prevent download of the whole database (it can be huge, let's hope Smile ) I think we have to use at least one parameter for the primary key: getcat.php?id= or maybe second for the Parent ID getcat.php?parent=

The protocol from the client side will look like this: (it is a draft, please think over it to make full specification)

1. The client sends to the server (using POST) the list with ID's of all articles it have in local database - script let's say "compare.php".

2. The server compares this list with it's own list in the table "Articles" and sends back two lists - one for the articles missing on the client computer and one for the articles missing in the server database.

3. The client using "getarticle.php?id=" script begins to download missing articles. When it is ready to save the article to the local database, it checks corresponding fields in the tables Categories and Authors. If the corresponding Category is missing, the client request it through "getcat.php?id=" and "getaut.php?id=".

4. Using the second article list, the client begins to upload the articles through "postarticle.php" using POST - here it is not very clear how the server will synchronize missing authors and Categories - maybe the client simply have to post whole information together - author, category and article...

We have to think little about security to avoid flood...but anyway I don't think the users have to be registered...

Regards
Post 01 Jan 2005, 11:49
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
decard



Joined: 11 Sep 2003
Posts: 1092
Location: Poland
decard
JohnFound wrote:
Happy new year! Smile

happy new year too! Wink

Quote:

Only one request for getcat.php. In order to prevent download of the whole database (it can be huge, let's hope Smile ) I think we have to use at least one parameter for the primary key: getcat.php?id= or maybe second for the Parent ID getcat.php?parent=


sure, I can make it.

Quote:
The protocol from the client side will look like this: (it is a draft, please think over it to make full specification) (...)

that's no problem from the server side. Just it would be cool if we could find a volunteer that will take care of client application... I don't have enough time for that... Anybody? Wink

regards
Post 01 Jan 2005, 13:18
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3502
Location: Bulgaria
JohnFound
decard wrote:

that's no problem from the server side. Just it would be cool if we could find a volunteer that will take care of client application... I don't have enough time for that... Anybody? Wink
regards


Well, I have some advance with client application - connections with SQLite and GUI. But I have no any experience with network applications, so any help will be highly appreciated. Smile

Regards
Post 01 Jan 2005, 16:06
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
decard



Joined: 11 Sep 2003
Posts: 1092
Location: Poland
decard
I found really clean description of HTTP protocol: http://www.w3.org/Protocols/HTTP/HTTP2.html. Also, Quetannon example can be really helpful, both as an example of networking application and as a protocol test.
Post 01 Jan 2005, 16:56
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3502
Location: Bulgaria
JohnFound
Decard,
Well, I read some things about sockets and HTTP protocols (thanks for the links) and I am almost ready to test some parts of the client with the server.
Please, only change the separator in the list from space to "|" (or any other that is not normally used in title or description text), because I can't parse the text with space separation.
Also, maybe we have to include some identifier to the output text in order to be possible to check for server errors for example without complex parsing.

Regards.
Post 01 Jan 2005, 22:02
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3502
Location: Bulgaria
JohnFound
Ah, also, why not to send integer values simply as a raw binary data. Do you think it is a good idea?
Post 01 Jan 2005, 23:47
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
decard



Joined: 11 Sep 2003
Posts: 1092
Location: Poland
decard
JohnFound wrote:
Please, only change the separator in the list from space to "|" (or any other that is not normally used in title or description text), because I can't parse the text with space separation.

yeah, I thought about it just after I uploaded it to the server... now it encloses both category name and description in double quotes. Do you think that separating all fields with (eg) | is better?

Quote:
Also, maybe we have to include some identifier to the output text in order to be possible to check for server errors for example without complex parsing.


I don't get it. What do you actually mean?
Post 01 Jan 2005, 23:53
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3502
Location: Bulgaria
JohnFound
decard wrote:
... now it encloses both category name and description in double quotes. Do you think that separating all fields with (eg) | is better?


Well, it seems to me it will be easyer for parsing, I will not change the char I search for. But quoting is OK too.

Also I though, whether it is better to send not ascii text, that have to be parsed, but directly binary data:
Code:
ID dd 1234
ParentID dd 0
Title db 'sdfhsd', 0
Desc db 'sdfuhsd', 0
  db 0 ; end of the record
ID.... ; the next row for multirow data.
    


At least why to convert ascii->integer->ascii all the time?

Quote:
Quote:
Also, maybe we have to include some identifier to the output text in order to be possible to check for server errors for example without complex parsing.

I don't get it. What do you actually mean?


Well, I mean, in some cases, the server will return html pages with errors, instead of your Category data. Also data can be corrypted in many ways. How to detect invalid data? The first way is to check the "syntax" of the response - whether there are spaces (whatever) between numbers, whether strings are quoted or not, etc, etc.
Second way is to use some headed - for example the first row of the text is "FADE CATEGORY" (it is possible for binary variant too - we can even send "FADE" - one dword and second dword, check sum of the remaining data in the message. Then with very good reliability the client can parse data without any check.

Regards
Post 02 Jan 2005, 00:06
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3502
Location: Bulgaria
JohnFound
To all:
I just made new page in Fresh site: Related projects.
The current FTnT work files can be downloaded from here: http://fresh.flatassembler.net/dm.php?file=ftnt

Regards.
Post 02 Jan 2005, 01:04
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2, 3, 4  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on YouTube, Twitter.

Website powered by rwasa.