flat assembler
Message board for the users of flat assembler.

Index > Heap > Firefox 32bit compilation

Author
Thread Post new topic Reply to topic
Enko



Joined: 03 Apr 2007
Posts: 678
Location: Mar del Plata
Enko
http://www.ghacks.net/2011/12/13/firefox-suffers-middle-ages-bloat/

I read this in some places.
It takes more than 3gb of memory to compile firefox, so they are having problems to build the 32bit version. Rolling Eyes
Post 17 Dec 2011, 13:58
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17249
Location: In your JS exploiting you and your system
revolution
They should use assembly optimisations instead of the "Profile-Guided Optimisation". And as a bonus they will get more than the paltry 10% speed improvement they get from automated tools. Razz
Post 17 Dec 2011, 14:16
View user's profile Send private message Visit poster's website Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
AFAIK, they don't use PGO/LTCG just for the speed benefit, but also to get code ordered in their executable according to load/execute order - which supposedly makes quite a difference wrt. load speed (firefox has often been criticized for slow load speed).

They're getting OOM because of the big and monolithic XUL.DLL - which used to be split into multiple separate modules, but was merged back together to the current lumping monstrosity because of said load speed problems.

Trying to do that ordering manually seems like a lesson in futility for an application as big as a modern web browser Smile
Post 17 Dec 2011, 15:22
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17249
Location: In your JS exploiting you and your system
revolution
Hmm, since Mozilla have identified that they have a speed problem then certain actions should have taken place. And IMO they should not have taken the lazy path of automated tools to "solve" the problem for them.

When I look at my FF folder on disc it shows 26.3MB, 218 files. I can scan (copy to nul) all those files in less than 0.5 seconds on my slow laptop. But FF takes around 10-15 seconds to start. That is a huge difference. Clearly it is not HDD loading time that is the bottleneck here. I suspect it is the paradigm being used, i.e. DLLs and all the associated binding requirements.

IMNO (in my naive opinion) I think that static binding would cut the load time to half. Still not perfectly optimal, but certainly more palatable.
Post 17 Dec 2011, 15:50
View user's profile Send private message Visit poster's website Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
Well, you cannot compare just loading files into memory with executable loading - it's done on-demand (it's using memory-mapped files internally), iirc it loads chunks of 64kb rather than 4kb though. So if you have an executable with 'random' code flow, you get very 'random' (as in lots of seeks) disk I/O... and we all know that HDD drive performance quickly becomes pathetic if you aren't doing linear access Smile

One could pre-scan the executables to ensure it's all in disk cache, and that might be a good idea for a plugin - but obviously that's somewhat symptomatic treatment, and I'm not sure I like the idea of <b>core</b> versions of software doing such tricks. PGO/LTCG reordering is a bit less dirty, even though it's a bit less effective. And while it's still symptomatic treatment, at least it's something you get "for free".

I do agree that executable load is only part of the sluggish load speed - Chrome (16.0.912.63), in addition to it's 1meg main executable, ships with ~64meg assorted DLLs and executables in it's Application folder, and it loads lightning fast. Firefox 8.0 clocks in at around 16MB.

Anyway, in fairness, a fresh install of Firefox with no plugins also loads very fast (these days - there were some versions where it was a bit sluggish), it's once you start adding lots of addons it slows down... I haven't looked much into what exactly causes the slowdowns, but it did help moving my firefox profile (not the executables) to a RAM drive.
Post 17 Dec 2011, 18:33
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
I have a speculation, that the slow loading of FF is because of some initialization code, not the loading itself.
Post 17 Dec 2011, 18:42
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
typedef



Joined: 25 Jul 2010
Posts: 2913
Location: 0x77760000
typedef
The best webs browser today is COMODO Dragon by COMODO. Made off the chromium level (like Google Chrome)but has more privacy which is a big deal to people like revolution.

http://help.comodo.com/topic-120-1-279-2524-Comodo-Dragon---Introduction.html

PS: I'm using it right now, it's fast.
Post 17 Dec 2011, 20:12
View user's profile Send private message Reply with quote
Enko



Joined: 03 Apr 2007
Posts: 678
Location: Mar del Plata
Enko
I didn't use COMODO browser , but I heard lots of good comments about it.


Its seems like Chromium had turned off PGO a year ago and are building 32bit version under a 64bit OS.
https://code.google.com/p/chromium/issues/detail?id=21932

Wich compiler use firefox to build windows versions? MSVCpp?
Why this one is not so popular? (Intel cpp compiler)
http://software.intel.com/en-us/articles/intel-compilers/
Post 17 Dec 2011, 20:48
View user's profile Send private message Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
Intel C++ compiler is relatively costly.

Microsoft Visual C++ has a free 'express' edition.
Post 17 Dec 2011, 21:16
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17249
Location: In your JS exploiting you and your system
revolution
I look at this as something similar to the sorting algorithm problem.

The FF design paradigm (like almost all C/C++ projects these days) can be likened to the bubble sort: good and simple for smallish stuff but once you get beyond a certain size becomes very slow and unwieldy. And no matter how much you optimise the code (or the loading order, or whatever) you are still stuck with a bad overall design that can never give any real performance benefits.

So what the FF team need to do now is change the paradigm and move to something more efficient for larger projects. I don't really know what that would be for a C/C++ project but I would expect that static binding (like I mentioned above) would be part of the new paradigm. And no doubt a few more things also.

I wouldn't really consider expecting them to move to assembly (I know there is much resistance inside the HLL camps) but for certain parts of the code it might make a lot of sense. Even at the cost of having to maintain different codes for different CPUs there can still be an overall win, it just takes a bit of understanding and the rewards can be not-insignificant.

BTW: I don't really care about a 15 second loading time. I am not complaining in any way. I am just following through with what the FF team has identified: that they want/need to improve the loading time.
Post 18 Dec 2011, 02:14
View user's profile Send private message Visit poster's website Reply with quote
typedef



Joined: 25 Jul 2010
Posts: 2913
Location: 0x77760000
typedef
I'd love to help them code in Assembly.
Post 18 Dec 2011, 11:08
View user's profile Send private message Reply with quote
ctl3d32



Joined: 30 Dec 2009
Posts: 204
Location: Brazil
ctl3d32
so would i
Post 18 Dec 2011, 11:17
View user's profile Send private message Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
revolution wrote:
So what the FF team need to do now is change the paradigm and move to something more efficient for larger projects. I don't really know what that would be for a C/C++ project but I would expect that static binding (like I mentioned above) would be part of the new paradigm. And no doubt a few more things also.
When you use static binding in this context, what do you mean exactly? Abolishing the use of DLLs and generate one huge executable?

revolution wrote:
I wouldn't really consider expecting them to move to assembly (I know there is much resistance inside the HLL camps) but for certain parts of the code it might make a lot of sense.
Dunno - most of the time, the browser isn't CPU bound, but suffers from some disk I/O sluggishness... code optimizations matter for some parts of the browser - but they're on that. Lots of work has gone into the JavaScript engines(1) of modern browsers, and work is going on for the compositing engines(2).

For (1) I'm sure that some people here would rather see the death of JavaScript, but that's just not going to happen - so the pragmatic thing is trying to make the experience as little painful as possible.

For (2) it's really something that should be handled by the GPU, so assembly doesn't buy you a lot here.

_________________
Image - carpe noctem
Post 18 Dec 2011, 14:13
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17249
Location: In your JS exploiting you and your system
revolution
f0dder wrote:
When you use static binding in this context, what do you mean exactly? Abolishing the use of DLLs and generate one huge executable?
Yes. Why is that such a big deal? The DLL's are all being loaded anyway so just do it all at one time and be done with it.
f0dder wrote:
Dunno - most of the time, the browser isn't CPU bound, but suffers from some disk I/O sluggishness... code optimizations matter for some parts of the browser - but they're on that. Lots of work has gone into the JavaScript engines(1) of modern browsers, and work is going on for the compositing engines(2).
But the disk I/O I saw was less than 0.5 seconds to read all the files, even the plugin and data files.
f0dder wrote:
For (1) I'm sure that some people here would rather see the death of JavaScript, but that's just not going to happen - so the pragmatic thing is trying to make the experience as little painful as possible.
Indeed I am all in favour of having JS sped up (for those that use it).
f0dder wrote:
For (2) it's really something that should be handled by the GPU, so assembly doesn't buy you a lot here.
At start up I doubt there is any compositing. In fact there is not even a window until the very last moment. FF is either waiting for something (network?) or it is computing something unseen.
Post 18 Dec 2011, 14:21
View user's profile Send private message Visit poster's website Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
Quote:
FF is either waiting for something (network?) or it is computing something unseen.


IMHO, it runs some initialization code and most probably this code is not compiled, but interpreted in JS. Although, I am not sure who makes the bottleneck - FF or Gecko. For example, K-Meleon is not radically faster than FF, despite it is native application.
Post 19 Dec 2011, 07:31
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
typedef



Joined: 25 Jul 2010
Posts: 2913
Location: 0x77760000
typedef
So, who wants to code the first web browser in Assembly ?

Let's all, the FASM members make the first one. What do you guys think ?
Post 19 Dec 2011, 10:37
View user's profile Send private message Reply with quote
Frank



Joined: 17 Jun 2003
Posts: 100
Frank
According to this blog entry (glandium.org), the slow Firefox startup is due to a huge number of relocations that must be done by the dynamic linker: " [...] while the kernel reads ahead by 128 KB chunks, when your relocation section exceeds that amount by more than an order of magnitude, it means I/O goes back and forth a lot between the relocation section and where the relocations take place."
Post 19 Dec 2011, 10:39
View user's profile Send private message Reply with quote
typedef



Joined: 25 Jul 2010
Posts: 2913
Location: 0x77760000
typedef
That reminds me of this funny article: http://www.wheaty.net/ADayInTheLife.htm
Post 19 Dec 2011, 10:51
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3500
Location: Bulgaria
JohnFound
Quote:
Let's all, the FASM members make the first one. What do you guys think ?


KolibriOS community will declare you a prophet. Very Happy
Post 19 Dec 2011, 12:31
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
revolution wrote:
f0dder wrote:
Dunno - most of the time, the browser isn't CPU bound, but suffers from some disk I/O sluggishness... code optimizations matter for some parts of the browser - but they're on that. Lots of work has gone into the JavaScript engines(1) of modern browsers, and work is going on for the compositing engines(2).
But the disk I/O I saw was less than 0.5 seconds to read all the files, even the plugin and data files.
Again, your linear-reading example - which isn't the access pattern used by the common operating systems. Perhaps it should be, though - since code is generally a small part of programs, compared to data. But then there's the crazy people that stuff all their resources into a single gigantic executable, and you wouldn't want to preload the entirety of that Smile

revolution wrote:
revolution wrote:
f0dder wrote:
When you use static binding in this context, what do you mean exactly? Abolishing the use of DLLs and generate one huge executable?
Yes. Why is that such a big deal? The DLL's are all being loaded anyway so just do it all at one time and be done with it.
It's a big deal because it'd be even more impossible to do LTCG+PGO Smile, and I do wonder just what it would buy you. Yes, you'd save some space on import tables, and even with binaries linked to a DLL version of libc there might be some slight amount of stub code linked in each PE... but I doubt you're going to save large quantities of disk space doing so.

Then there's possible CPU savings from not having to hook up import tables. I haven't attempted to benchmark how much time is spent on that, but the PE loader does binary search through exports, so it's not too shabby. I did wonder why the firefox imports weren't bound, but I guess import binding might be useless in these days of ASLR.

revolution wrote:
f0dder wrote:
For (2) it's really something that should be handled by the GPU, so assembly doesn't buy you a lot here.
At start up I doubt there is any compositing. In fact there is not even a window until the very last moment. FF is either waiting for something (network?) or it is computing something unseen.
Indeed, it's not relevant to the startup speed issue, but it's something that matters for speed in general Smile

Frank wrote:
According to this blog entry (glandium.org), the slow Firefox startup is due to a huge number of relocations that must be done by the dynamic linker: " [...] while the kernel reads ahead by 128 KB chunks, when your relocation section exceeds that amount by more than an order of magnitude, it means I/O goes back and forth a lot between the relocation section and where the relocations take place."
Ouch, seems even worse than Windows. At least the Windows PE relocations are stored in a 'compressed' format, arranged by page... which also means relocations can be applied as sections of the executable are paged in, and without marking the code memory pages as dirty.

_________________
Image - carpe noctem
Post 19 Dec 2011, 15:17
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar.

Powered by rwasa.