flat assembler
Message board for the users of flat assembler.

Index > Heap > [contest] database, maybe ascii text flat file is better?

Goto page Previous  1, 2, 3
Author
Thread Post new topic Reply to topic
sleepsleep



Joined: 05 Oct 2006
Posts: 8868
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 334455
sleepsleep
revolution wrote:

But also, remember that notepad is a very restricted editor. If your DB is larger than your available RAM then there is no way you can edit it in notepad. Can you always guarantee that your DB will be smaller than your available RAM? IIRC notepad will convert ASCII files upon loading and uses two bytes of RAM per single ASCII character. Plus inserting/deleting text with notepad in large files takes a long time.

well, notepad or any other ascii editor that able to display ascii text as if like using notepad.

hex editor is okay, as long as, the data are displayed in a way that we can read/write/modify it like using notepad.

Quote:

It would be pretty tricky to edit CRCs in notepad manually. I suppose it could be stored as hex, but calculating it is not so friendly.

it could be by manually placing a "mark" on that row after we clear the corrupted part to let the reader know the row is checked and it should calculate a crc and mark that row ok or any other method that you find easy and simple.
Post 25 Aug 2011, 22:29
View user's profile Send private message Reply with quote
pabloreda



Joined: 24 Jan 2007
Posts: 99
Location: Argentina
pabloreda
ok this is the app

http://www.mediafire.com/?k2383dpjsy76qwk

only sorting and filter, the source code in my google code in short

I found a bug !!! never use with 90000 records !!

well in time to sort and filter.. try !!

the program is INTERPRETED by r4.. sure a asm code can speed !!
Post 25 Aug 2011, 23:58
View user's profile Send private message Visit poster's website Reply with quote
AsmGuru62



Joined: 28 Jan 2004
Posts: 1408
Location: Toronto, Canada
AsmGuru62
revolution makes a good point.
I just tried NOTEPAD on the file I made from sleepsleep source and WinXP says: "use another editor".
Post 26 Aug 2011, 01:45
View user's profile Send private message Send e-mail Reply with quote
pabloreda



Joined: 24 Jan 2007
Posts: 99
Location: Argentina
pabloreda
I use ConText for .txt editing... great soft, read the file without problem
http://www.contexteditor.org/

I upload the sources
http://code.google.com/p/reda4/source/browse/trunk/r4/Apps/stresstest.txt
Post 26 Aug 2011, 02:06
View user's profile Send private message Visit poster's website Reply with quote
Madis731



Joined: 25 Sep 2003
Posts: 2141
Location: Estonia
Madis731
I think there are things like 'self-recovering' databases. I think you can add as much redundancy as you like to a DB file (simple copy paste for starters) and sync them without user interference.

Even if we assume DB less than 4GB (32-bit addressable range, most common RAM-configuration today), that 4GB can be copied in under 8 minutes on a slow HDD.

However, plain-text database is usually larger than similar SQL-based one. Plain-text databases can corrupt as easily (someone deletes the unlocked *.txt file or writes 0000000... into it).

If you want to backup your plain text-based database, then you'll need more time and HDD space than with SQL DBs.

I do agree that ascii flat file is intriguing and I can see from responders that it has been used successfully. I hope that there are some other ways too, like instructing SQL database to store its data in a readable form. It would then combine the power of SQL language and the simplicity of reading/recovering/writing of database data.
Post 26 Aug 2011, 06:41
View user's profile Send private message Visit poster's website Yahoo Messenger MSN Messenger Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 8868
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 334455
sleepsleep
pabloreda wrote:

ok this is the app

http://www.mediafire.com/?k2383dpjsy76qwk

only sorting and filter, the source code in my google code in short

I found a bug !!! never use with 90000 records !!

well in time to sort and filter.. try !!

the program is INTERPRETED by r4.. sure a asm code can speed !!

it looks COOL Smile
but why the application constant using processing power around 8% to 15%? is it because of the GUI?

i use "D:\MY\Desktop\stresstest>r4 db\test\a.fk" to run it.

Image
Post 26 Aug 2011, 11:40
View user's profile Send private message Reply with quote
ronware



Joined: 08 Jan 2004
Posts: 179
Location: Israel
ronware
sleepsleep: Why not use a very small and fast SQL engine like SQLite?

Why you should consider it -

1. very robust code, used by many thousands of products
2. free (!)
3. fast
4. ability to dump database as plain text SQL
5. easy to use

Reva has included an interface to SQLite for a very long time, it uses SQLite as the format for the help system.

Why you should not write your own database:
1. easy to make bad errors (which will lose your company data)
2. distract from writing the actual application (takes a long time to write a robust database layer, and that is not your real problem you want to solve)

If your employer doesn't care that you spend your time writing something when you don't have to, or if you want to experiment on your own time, that's fine of course...
Post 26 Aug 2011, 11:52
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 8868
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 334455
sleepsleep
Quote:

sleepsleep: Why not use a very small and fast SQL engine like SQLite?

i google "sqlite corrupt" and got about 331,000 results

i know all the advantages of using something that already established especially sqlite, but i want something that if it is corrupted, then a notepad or any other ascii editor is enough to repair stuff.

i don't want to depend / google for tools to fix corrupted db and since the input data rate is 100 records per 1 hour and they work around 9 hours, so let say 1k records per day, 31k per month, 372k per year,

Quote:

Why you should not write your own database:
1. easy to make bad errors (which will lose your company data)
2. distract from writing the actual application (takes a long time to write a robust database layer, and that is not your real problem you want to solve)

yeah, true, but i think this world need a http data text file writer reader, for small enterprise usage, not big data.

Quote:

If your employer doesn't care that you spend your time writing something when you don't have to, or if you want to experiment on your own time, that's fine of course...

i didn't work, i just freelance provide technical support for several companies around my areas, they phone me, then i fix their problem, then they pay me.
Post 26 Aug 2011, 12:12
View user's profile Send private message Reply with quote
ronware



Joined: 08 Jan 2004
Posts: 179
Location: Israel
ronware
I've used SQLite for years, and never experienced any data corruption.

Anyway, if you are talking about multiple users at the same time, you should definitely not use a plain-text database, nor should you use SQLite -- but rather, some transactional database (like MySQL, which is free and very robust).
Post 26 Aug 2011, 12:25
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger Reply with quote
Madis731



Joined: 25 Sep 2003
Posts: 2141
Location: Estonia
Madis731
...AND like ronware mentioned - you can "DUMP PLAINTEXT SQL"!!!
You can do it at daily intervals and when something corrupts, you can just import it. Voila!

If your dump gets errors, you can fix them in notepad Smile
Post 26 Aug 2011, 13:10
View user's profile Send private message Visit poster's website Yahoo Messenger MSN Messenger Reply with quote
ronware



Joined: 08 Jan 2004
Posts: 179
Location: Israel
ronware
Precisely! And that is what I do in fact ...
Post 26 Aug 2011, 13:13
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 8868
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 334455
sleepsleep
i know,
but why couldn't we just design one for low transaction data input, text file so as to appreciate how the big sql works.
Post 26 Aug 2011, 13:22
View user's profile Send private message Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 8868
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 334455
sleepsleep
i revive this contest with USD 10 paypal prize =) atm,

might increase prize depend on participation!

maybe binary database is ok too, easier to calculate, i mean, something like dbase style. fixed width,

file start
database properties,
index pointer
data
1 to 4000 or etc, depend
index pointer
4001 to etc

store data in binary, then we use hexedit also can modify it easily,

what you guys think?
Post 18 Feb 2013, 09:08
View user's profile Send private message Reply with quote
OzzY



Joined: 19 Sep 2003
Posts: 1029
Location: Everywhere
OzzY
I just use Sqlite when I want deployability of text files, but power of SQL.

Or maybe you should try Metakit if you don't like SQL: http://equi4.com/metakit/
Post 18 Feb 2013, 13:32
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2, 3

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar.

Powered by rwasa.