In context of SQLite discussion on this board, there was once mentioned the idea of some similar project written in assembly. This idea was interesting for me from the beginning, as for some of my projects it would be a great thing to have something like that. Even more: I'm thinking about making some database engine written in assembly with an binary-level access language, which would have then some external wrapper to support SQL syntax, but itself would perhaps be more efficient and easy to use from assembly language programs than text-processing-based SQL (which is better for scripting languages like Perl of PHP). The other nice thing would be to make it portable between any x86-based systems just like fasm is.

Myself I may not be able to start and maintain such a complex project, but perhaps by cooperating we would be able to embody it. Therefore I ask here to let me know if anyone's is interested in cooperating in such project, or has some interesting ideas.

Also, if you've got some interesting links about database systems design and algorithms, please post them here.

This is very, very, very funny for you to post, for a reason.

Yesterday, I got the idea of making a database OS, for the various advantages that such an OS would bring. I realised that the way current databases work would make it hard for such an OS to be efficient, so obviously I'd have to do two things:
- Devise my own database design, so that it is able to work with an OS efficiently.
- Actually make a database implementation, with the OS built-in.

Well, I spent all of yesterday designing the database mechanics and I must say it looks promising enough to warrant a try.

So, today, I was going to start designing how everything should be stored in memory when suddenly I had an idea: what if I made the database flexible in its uses, and rather than use it merely for an OS, make it usable for other things? It was already extremely flexible because of the database design I had made, but I knew that for it to be used as other things, being mechanically flexible wasn't enough: it'd have to be binarily flexible. (is binarily a word?)

This soon brought me to a second goal:
- The database has to be OS independant.

Of course, something can never be 100% OS independent, but I think I budged along quite well. I sort-of devised a plugin system, by which almost everything in the database system can be controlled by a plugin.

This way, porting from one OS to another is basically a matter of changing the right plugins and formatting the database executable. It is also possible to load plugins durin run-time.

[size=small]
Basically, the way the plugin system works is:
- Each plugin has a name associated with it. The name must not exceed 4 byte-length and therefore can be stored on a simple dword (and passed on the stack: always a plus).
- The database system looks for a plugin with the right name, and calls one of its functions, which are also indexed by 4-byte names,

That's rather simple, isn't it? Well, the problem is obviously "how am I going to act as a pseudo-kernel for plugins if I don't have access to the manipulation of virtual addresses?" Basically, I copy the whole plugin into a certain address (0x10000 at the moment). They are assigned the access of 0x10000 to 0x1FFFF to themselves, whilst 0x20000 upwards being shared memory. (up to a maximum of 0xFFFFF, at the moment.)

These values are good because it would allow the database to be portable even to 16bit systems without too much hassle, although that's a bit more drastic than changing between OSes. [/size]

That was a bit besides the point, but it serves to illustrate how far I'm willing to optimize a system (the 4-byte thing, that is).

Now, I have already dedigned a method of querying the database in a text-processing way but, as you point out, this is a saturated market. I'm sure that designing a binary query of sorts is perfectly possible (and necessary, too). I will deviate my attentions to the design of such a binary system. Last, but not least, you're free to join me in my adventure if you want. I sent a project application to sourceforge.net. The unix-name of the project is dbsystem

Cya there. (If the project gets approved.)

I think we can start simple INSERT / DELETE. Hmm, I feel like I've suggested this before, but here me out:
SELECT would of course be very neccessary, but that is it. UPDATE would be like a macroinstruction and WHERE would be a macro of multiple SELECTs. When we have it working, we can improve it further, by adding indexes, rewriting syntax for more optimized approach and implement algorithms that we may accidentally find on the net. I had a relatively thorouugh course in DBs in the university so I can give you some hints on the way like about the different types of indexes and their uses...
...I myself am busy with other projects, but I'll see how I can help this dbengine.flatassembler.net to become reality

I'm currently looking into hashtables and seeing if they are at all useful to db.



Might not make much sense now,but it'll probably be clearer later on.

Tomasz (and interested on the project), this database will be a relational database or an object oriented database?

http://en.wikipedia.org/wiki/Object_database

To make this project mature could be take too long time so I think it's important to see which kind of databases will be used in the future (RDBMS or ODBMS).

http://www.odbms.org/introduction.html

Regards

Wich one in your opinion the assembly language is more suitable for?

Well I'm not a DB expert but seems to be O-O more suitable for assembly, since you make queries using template objects seems that you don't need to send text (or opcodes as your idea).

However, O-O languages implements objects quite diferent each other so you will need to create multiple interfaces instead of a single one (SQL).

Hope someone with more knowledge answer you question more accurately

Regards

Some other nice idea that I once saw on this board was to try rewriting SQLite in assembly - that would be perhaps at least instructive.

Since SQLite is an embeddable database system it's more suitable to give it a try converting it to ODBMS since it shares the same memory space that the client.
ODBMS.ORG says in When to Use an ODBMS this:

Of course I understand that an assembly language programmer usually dislikes write in OO way assembly code but I think that FASM is powerful enough to write friendly macros, struct macro already supports inheritance, it just need some improvements to support method binding and allow virtual methods (polimorphism).

The model i'm using for db isn't too far off from object databases, but is much more lose. I guess i'll release the documentation on the site once the thign gets approved.. (4 days in pending queue... is it really a queue?)

i'm also not an expert in db internal, but i think some general OOP support should be added as initial stage (embedded and/or via macros) (it should be nice not only for db)

Are we thinking of a db that will be used within general public or are we aiming at being different (take MenuetOS as an example) and write something totally different that others will hate at the beginning

I think doing something that others have done over&over&over again isn't too interesting. assembly style SQL sounds fun...at least for now:

...or maybe it will become too obfuscated. I hear a poll coming up.

I like Madis731's idea - ASM style SQL. Most importantly, it will be something new and unique!

YONG

Concerning the syntax, I think a 'middle way' would attract more people. The one that is suggested seems to be too confusing for complex statements and queries... I also would like to participate to the project if it is decided to be done (or at least start:). I think I can contribute in language parsing as I already have an experience and interest in that. Don't have much knowledge about the internals of a db engine though.

heh. well, i decided to abandon db because sourceforge takes ages and eventually i lost interest in said database. So, i'm interested in Madis's idea... care to elaborate a bit more on it?

Well, it was just an idea, but if enough people are interested, then we can think of a logical way to give asm programmers to express their way of thinking and their syntax in the garment of SQL (actually I meant vice versa )

i think there should be some c-call functions like

which can be automatically converted to macros like:

where "what","from" are some supported structures to explain wide parameters. [from] can be a direct handle to the table, etc.

it has to be started from single-table toy, then may be extended to more complex structure.

Yes, I think that giving it C-compatability would be a wise move. Very much so.

As for [what] and [from]...

couldn't we just do a



These would return a pointer to a table with all the selected entries.

You could have a function that merges tables together to make another table:


This way, '_mselect' is just a selection on a merger of tables.

To make tables from scratch...



Makes a table prototype.



Would add an element to the prototype.



Would create a table from the prototype. Now you can add entries to the table, but this can be figured out later...

....

Advantages of this system? Well, it works perfectly with C, which is portable, doesn't require the use of strings (but you *can* use strings), the structure of entries can match a C structure (or an asm one for that matter (use virtual)). IT would be faster than SQL by a lot, I would think. But then again, it doesn't REALLY match asm as such.

Mota, with "ccall _select, table, evaluative_func " do you mean that the database engine will call evaluative_func procedure? In such case that will be not work very good with a server database. Another problem is since the database engine can't know what evaluative_func does it will be forced to iterate all the table's records.

Note that is preferable to spend time in parsing a SQL statement than make one disk read access, CPUs can do a lot of work in a few milliseconds.

Regards

the idea is to make sql-compatible set of functions. so, these functions could be used directly on local base or server, but can be easy used to parse sql statement. anyway sql wil be needed, if you plan to give your application to user and let him make changes without recompilation of main module.