flat assembler
Message board for the users of flat assembler.

Index > Main > STDIN/STDOUT to/from memory buffer; Battle FASM vs C++

Goto page Previous  1, 2, 3, 4  Next
Author
Thread Post new topic Reply to topic
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
Quote:
nothing unusuall about it so should be accept...


Thanks dancho. That was my assumption as well.
My opponent have great problems with the speed, so now he is implementing hash search in the links list in hope it will help. In vain, of course, the problem is not there - tests with file with no links at all show the same poor speed.

On the topic: Because my opponent protested the use of GetFileSize for reading STDIN (it can't read pipes, for example "type somefile.txt | markdown > output.txt" fails) I reverted to the first variant - reading chunks with memory reallocations.
This of course slowed down my program from 0.6 to 0.8s for 8Mbytes file. Compared with his 56s it is not relevant for now. We will see the next battle round. Smile
Post 06 May 2012, 10:07
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
dancho



Joined: 06 Mar 2011
Posts: 74
dancho
excellent score John,really excellent,
since you arent using GetFileSize Im just wondering what kind of mem allocation you used,VirtualAlloc + MEM_RESERVE + MEM_COMMIT , or HeapAlloc + HeapReAlloc ?
/* well there is one more method with CreateFileMapping + SEC_RESERVE + SEC_COMMIT but that one is probably slowest one */
Post 06 May 2012, 14:33
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
I use FreshLib, and it uses Heap functions in Win32 and libC malloc in Linux.
This disastrous result (70..80 times faster) is not exactly my win, but IMHO, a fault of the HLL programmers to see the problem from the proper angle. Note, that I made almost no effort to optimize for speed or for size. And recently even rejected some of the faster solutions...
Post 06 May 2012, 14:48
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
Enko



Joined: 03 Apr 2007
Posts: 678
Location: Mar del Plata
Enko
I took some time and read the thread. Actually what dvader clames is that the poor performance of his program is his fault and not of the compiler becouse he didn't implement hash tables.

For what I could estimate, some of this should be true, because he doesn't seems to be as skilled as JohnFound, but still, I don't think that what he claimed about HLL compiler is true. Because, asm program will use better the resources enabled by the Os.

Its like calling printf in assembly and c, using fasm, it wouldn't load all the c runtime crap (like heap checking wich could be desablead) and other stuff.
Post 07 May 2012, 23:16
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
The problem of dvader's program is not only in the hash tables. I specially ran some tests with only one link. In this case the linear search is even faster than hash table search. The performance in this case is also very poor.

My statement here is that the poor performance is not because of the particular programmer skills, but because of the way HLL programmers solve problems.
Post 08 May 2012, 03:52
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
typedef



Joined: 25 Jul 2010
Posts: 2913
Location: 0x77760000
typedef
JohnFound wrote:
My statement here is that the poor performance is not because of the particular programmer skills, but because of the way some HLL programmers solve problems.
Post 08 May 2012, 04:21
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
"...most HLL programmers solve problems."

Note that dvader is not a bad programmer at all.
The fact he managed to make 4k useful program in OOP C++ is impressive for me.
On the other hand, my program is actually not very good. It is not optimized in any aspect. Neither for speed, nor for size. Especially the parser itself. I just made it to work somehow.
So, what is the problem? Over-trust to the HLL compiler?
Whatever is the reason, is general problem.
Of course it is possible to make pretty good programs in HLL, but IMHO, it is despite of HLL, not because.
Post 08 May 2012, 05:32
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
Too bad I can't read Bulgarian Smile

Are there any rules set for the competition - both can-dos and can't-dos (and how much of markdown syntax to support, plus error-handling?), but also how the results are ranked? Are there any big-fscking-markdown-file generation apps posted so there's something realistic to test against?

IMHO Dvader is approaching this wrong - trying to do too much in one iteration. The sensible approach would have been to create something working that uses standard C/C++, then focus on making it fast... and finally, if that's a goal, make it small.

_________________
Image - carpe noctem
Post 08 May 2012, 05:38
View user's profile Send private message Visit poster's website Reply with quote
Enko



Joined: 03 Apr 2007
Posts: 678
Location: Mar del Plata
Enko
Quote:
The fact he managed to make 4k useful program in OOP C++ is impressive for me.

Its actually posible make it even smaller by not using the standard runtime.
Quote:

On the other hand, my program is not optimized in any aspect. Neither for speed, nor for size.

Usually in assembly you get smaller executables without even trying it. Its plain normal.

Quote:

"...most HLL programmers solve problems."

And YES, I will agree on this in some points.

Its because of the way you try to solve somthing in hll...
Sometimes the more "simplistic" solution, the one that takes less lines, but much more complicated structures is used.
In this case, the code looks preaty, it's easier to read, to understand, to update and this is considered I think a good practice. But this type of code, will some times generate lots of junk code... so there's the point.


While on assembly, every line you write, is in fact what the cpu will execute. While you try to write less code, it will probably generate less instruction. Thus, the program is more optimized.


What I mean in few words:
HLL, less code >> doesn't guarantee less cpu instructions, or less size.
ASM less code>> less size, les instructions, thus more speed.
Post 08 May 2012, 05:50
View user's profile Send private message Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
Enko wrote:
Quote:
The fact he managed to make 4k useful program in OOP C++ is impressive for me.

Its actually posible make it even smaller by not using the standard runtime.
He's not using the standard runtime, and IMHO trying to get executables under 4k size is mostly a curiosity anyway - sure, it's cute employing all kinds of tricks, but most interesting stuff is going to be larger than 4kb anyway... and then there's stuff like filesystem cluster size and memory pagesize to consider as well Smile

Enko wrote:
While on assembly, every line you write, is in fact what the cpu will execute. While you try to write less code, it will probably generate less instruction. Thus, the program is more optimized.
For size, yes... but faster code comes at the price of size - loop unrolling, anyone?

_________________
Image - carpe noctem
Post 08 May 2012, 06:35
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
f0dder wrote:
Too bad I can't read Bulgarian Smile


The google translator make some kind of funny but better-than-nothing translation. Of course some of the details will be lost.

Quote:
Are there any rules set for the competition - both can-dos and can't-dos (and how much of markdown syntax to support, plus error-handling?), but also how the results are ranked? Are there any big-fscking-markdown-file generation apps posted so there's something realistic to test against?


The rules are pretty unrestricted. The task is to implement very small but still useful subset of markdown:
1. Paragraphs handling
2. inline markup with "*" - "this *is important*" should become "this <em>is important</em>"
3. Headings from H1 to H6 in the variant only with markup in the begin of the line: "### Heading of 3-th level"; without handling of the trailing "#"s.
3. Proper handling of CR, LF, CRLF and LFCR line ends.
4. Proper handling of UTF-8 source files.
5. Only external links, forward definition is allowed. Simplified link definition: "[name] address"
6. (it is not part of markdown) The parser should create table of contents, based on the headings and place it at the begin of the HTML file.
All, particularities are leaved at the discretion of the developer. For example, I am creating hierarchical TOC with numbered items, dvader makes it linear with bullets. It is OK for me. (if he manage to outperform me, I will strip this functionality of course).

Quote:
The sensible approach would have been to create something working that uses standard C/C++, then focus on making it fast... and finally, if that's a goal, make it small.


Well, IMHO, this is exactly what he is doing. But this "something working" became 80 times slower than my "something working" in assembly.
Now he is working on "making it fast", while I am hanging on the forums, talking about programming strategies.
But what about "faster HLL programming?" Wink

Enko wrote:
Usually in assembly you get smaller executables without even trying it. Its plain normal.
...
While on assembly, every line you write, is in fact what the cpu will execute. While you try to write less code, it will probably generate less instruction. Thus, the program is more optimized.

Yes, this is the statement I am trying to prove. Assembler creates more or less optimal programs by default, while HLLs make optimal programs only after serious effort by the developer and sacrificing all advantages of HLLs like readability, fast development and easy support.

_________________
Tox ID: 48C0321ADDB2FE5F644BB5E3D58B0D58C35E5BCBC81D7CD333633FEDF1047914A534256478D9
Post 08 May 2012, 06:37
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
Ah, I forgot to mention that there was implemented in Perl (by the user Sun) test generator, (attached to this post) that can create very big "our-markdown" test files.
In general, while I worked on the program, I used files with 100000 paragraphs and 1500 links (approx 80Mbytes), but the C++ version needed 10 times smaller files, so I use 8Mbytes test files now.
Post 08 May 2012, 08:05
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
JohnFound wrote:
3. Proper handling of CR, LF, CRLF and LFCR line ends.
What does "proper" mean in this context? Smile

IMHO it would mean being able to read the forms correctly, but not necessarily output the same type of linefeed. At least, how I understand markdown is that (preformatted blocks aside) you generate <p> output elements and thus the input linefeeds are just for prettification, and are discarded in the output.

JohnFound wrote:
4. Proper handling of UTF-8 source files.
Ah, that does complicate matters slightly Smile

JohnFound wrote:
f0dder wrote:
The sensible approach would have been to create something working that uses standard C/C++, then focus on making it fast... and finally, if that's a goal, make it small.

Well, IMHO, this is exactly what he is doing. But this "something working" became 80 times slower than my "something working" in assembly.
Nah, he's started with a bunch of his own (simple/naïve-and-small) classes instead of simply using libc/stl and then identifying bottlenecks. IMHO the focus on size here is a bit silly, but I understand it's part of the flamewar Wink

JohnFound wrote:
Now he is working on "making it fast", while I am hanging on the forums, talking about programming strategies.
But what about "faster HLL programming?" Wink
That would imply using the full platform - like, if you're doing Java development, you'd be insane if you didn't use the gazillion existing libraries instead of trying to roll your own, it's one of the major strengths of Java...

JohnFound wrote:
Ah, I forgot to mention that there was implemented in Perl (by the user Sun) test generator
Can you attach it here? Couldn't download it as an anonymous user when I tried earlier today.
Post 08 May 2012, 08:42
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
Quote:
IMHO the focus on size here is a bit silly, but I understand it's part of the flamewar

Well, my statement was defined exactly this way:

On equal functionality, assembly written programs are faster and smaller than HLL written programs.

This statement was impugned by dvader, so this is what we are trying to prove/disprove in our small battle/flamewar.

In the attachment is the last version of the generator used.


Description: Perl test files generator.
Use:
generator X Y > output.file
X - count of the paragraphs
Y - count of the links

Download
Filename: generator4.zip
Filesize: 4.56 KB
Downloaded: 92 Time(s)


_________________
Tox ID: 48C0321ADDB2FE5F644BB5E3D58B0D58C35E5BCBC81D7CD333633FEDF1047914A534256478D9
Post 08 May 2012, 09:10
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
So, the story have some progress. The Empire strikes back with new version.
dvader, declared, that he will copy the algorithms from my sources in order to achieve "fair competition".
After few days, he posted his new version, with great improvement in the speed. Now his program was only 2 times slower than mine.
BTW, I created source repository for his code in order to follow changes better: click here
Actually besides worse performance, his implementation have some bugs.

Today, I posted my answer - it was time for some code cleanup and optimizations. (the code is as always: here
I will attach the executables, if someone wants to make some tests.

The speed increased slightly, but the main problem again is the I/O from and to the disk. Now I think about asynchronous I/O, but it seems to be annoying for implementation. Smile


Description:
Download
Filename: markdown_asm.zip
Filesize: 13.93 KB
Downloaded: 96 Time(s)

Post 09 May 2012, 17:21
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
JohnFound wrote:
The speed increased slightly, but the main problem again is the I/O from and to the disk. Now I think about asynchronous I/O, but it seems to be annoying for implementation. Smile
Yeah, it's not easy doing it right when you're parsing stuff - things like line endings or UTF-8 sequences crossing buffer boundaries, eek. Been toying with the idea of using a sort of reactive state machine, been don't have the energy or time (and I'm too lazy Wink) to toy with it.

Interesting challenge, though Smile

_________________
Image - carpe noctem
Post 09 May 2012, 17:31
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
f0dder wrote:
Interesting challenge, though


Yes, and very edifying. This story illustrates how assembly and HLL programmers think about problem solving.
I only wish to collect information about more challenges like this, because one case is not very representative.
Post 09 May 2012, 17:37
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
Enko



Joined: 03 Apr 2007
Posts: 678
Location: Mar del Plata
Enko
I have an idea for the next chalenge. Loading a 24bit bitmap from disk and then displaying it on the screen.
The catch is that you can't use the LoadBitmap api.
Post 09 May 2012, 19:59
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
Enko, the main idea behind my battle was to create some really useful program, or as close as possible. That was the reason I wanted to not specify the particular algorithms, but the common behavior of the program and to leave the programmers free to search for solution.
It was not actually about C++ vs FASM, but C++ programmer vs FASM programmer.
The continuation was interesting - the C++ programmer said that he will copy the algorithms from my sources in order to get "fair competition".

The end of the story is pretty predictable, we will reach the limitations of the OS, my program (I hope) will be smaller and slightly faster on most tests (not all), the public will applaud the great C++ compilers and will not realize that dvader reached this result with greater efforts than my efforts.
Post 09 May 2012, 20:37
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
Enko



Joined: 03 Apr 2007
Posts: 678
Location: Mar del Plata
Enko
Quote:

The end of the story is pretty predictable, we will reach the limitations of the OS, my program (I hope) will be smaller and slightly faster on most tests (not all), the public will applaud the great C++ compilers and will not realize that dvader reached this result with greater efforts than my efforts.

I thought the main idea behind the HLL is to produce application faster. So if for the HLL programer it taked more time to produce the same result.....
Shouldn't it be the other way, write asm to optimize the speed of HLL. Not read ASM source to produce faster HLL program.
Post 09 May 2012, 23:16
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2, 3, 4  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar.

Powered by rwasa.