Firefox 3 Memory Benchmarks and Comparison
by Sam Allen
Web browser performance is an often talked-about and flaunted thing, but many claimsare not really backed up by solid evidence. I wrote software that collected millionsof data points over 14 hours of actual browsing time, and this article revealsmy findings.
ProblemMany people load hundreds of web pages, sometimes at the same time, oftenover periods of 3+ hours. Users complain about the memory usage of Firefox, Safari,or Internet Explorer, and we need a way to identify which browsers are better atmanaging memory than others. Traditional benchmarks do not look at all the thingsyou might do with a program, and we need real-world numbers over a period of hours.
SolutionI developed a Windows Forms application in .NET called Memory Watcher that "watches"the system memory numbers. It uses a timer to poll the processes every 3 seconds.It then records every number and also prints them out in a grid on the screen. Thisallows us to keep track of each program's memory usage over time and with real-worldusage.
Memory ProfilesThese results are from opening Memory Watcher and then using the browser between9,000 and 11,000 seconds (close to 3 hours). Each browser is tested in a separatesession, and there are brief periods of inactivity throughout the time period.The vertical axis is the memory used in MB, and the horizontal axis contains thememory "checkpoints" my program took (one every 3 seconds).
    
Benchmark DetailsThe above profiles are not a direct comparison in any way, but they offer a visualizationof trending in the memory behavior of the layout engines and interfaces.This is not a diagnosis or bug report. Let me show some important metrics of theabove results.
| Browser name | Exact version | Time active (s)
Hours | Comments | | Safari | 3.1.2 | 10,470 s
2.91 hours | Normal browsing | | Firefox | 3.0 | 9,681 s
2.69 hours | Normal browsing
No extensions | | Flock | 1.2.2 | 10,146 s
2.82 hours | Flock is based on Firefox 2.0
No extensions other than the default | | Opera | 9.5 | 9,855 s
2.74 hours | No extensions
Only browser was used | | IE | 8.0 | 10,236 s
2.84 hours | Used 7.0 rendering mode
No extensions |
The system is Windows Vista SP1, and the computer has 3.0+ GB of RAM. No pluginsare disabled, but the Acrobat Reader and Java plugins were (presumably) not used.Flock is based on Firefox 2.0 but its memory usage is probably worse because ituses built-in extensions.
- Just regular stuff
These aren't stress tests, and I probably never went over 4 windows in each browser,with at most 3 tabs in each window. I didn't look at many pages that are extremelyheavy on images, and no "browser benchmark" style pages. Gmail was used on eachbrowser. - Not just pages
It is hard for a regular benchmark to "simulate" a user actually clicking on things.Interactions with the user can greatly influence memory or performance. Having aresponsive browser is probably more important than just having a "fast" one at showingpages. - Plugins included
My profiles include Flash and possibly other plugins. A browser might have memoryissues with a plugin and that could cause a significant problem with the user experience.(Most Windows Vista crashes have been due to graphics cards, not Vista itself, forexample.) - Real-life usage
An automation script will never give the same insight into performance over timeas will this sort of profile. As developers, we want to make programs that workwell for our users, and not just for tests. The tests capture the "rhythm" of softwareusage.
Final Memory MeasurementsThe data in this article are those reported by Windows Vista, but the individualnumbers should not be compared to each other. Some browsers were tested slightlylonger than others, and some different pages were loaded. That said, here are thefinal performance metrics.
| Browser name | Ending private set in MB | | Safari | 636.9 | | Firefox 3 | 111.8 | | Flock (Firefox 2) | 191.9 | | Opera 9.5 | 190.6 | | Internet Explorer | 194.4 |
About the "Memory Watcher"Memory Watcher is a small program I wrote that records the memory usage of eachprocess on the system every three seconds. It uses the PrivateMemorySize64long value from the Process collection in .NET.
- Simple
There are tools similar to this, offered on every platform, but they are not usuallyeasy to use. Memory Watcher provides a super-easy way to monitor every process andsilently work in the background. - Exports to spreadsheet
It exports the currently viewed data to a CSV file. These data are easily taken into Excel, and were used for the graphs in this article. - Implementation notes
The application uses a DataGridView control,and sets its DataSource property to aDataTable which is built from the objectcollection. It uses a Timer to poll the system every 3 seconds. It offers searchingand filtering of processes using a TextBox.
ConclusionThese profiles are meant to provide a picture of what the memory behavior of popularbrowsers is over a period of time, not to provide absolute benchmark times. Firefox3.0 shows memory usage that is significantly lower than Firefox 2, which also doesvery well. Here is a summary of my results.
- Safari 3.1
Safari on Windows shows extremely poor memory management, and I do not know whetherit ever reaches a high water mark. If this is by design, it is certainly a designthat looks inefficient and seems to contradict Apple's marketing. - Firefox 3.0
This browser exhibits memory usage that is by far lower than the others. It releasesmemory to the system and the trend line is nearly flat.
(This is likely due to the efforts outlined here.) - Flock (based on Firefox 2.0)
Flock did very well and this browser and Firefox 2.0 could likely be run for longperiods without causing many problems. The extensions probably reduced the efficiencysomewhat. - Opera 9.5
Opera's performance was about as good as Firefox 2.0 (Flock), and it could likelybe used for very lengthy sessions. However, Kestrel is certainly not a revolutionaryor even notable technology in this arena. - Internet Explorer 8 Beta 1
IE did well in the profile, although a worrying trend in the data could indicatethat it would keep escalating. However, this browser could likely sustain many hoursof moderate usage.
Final ThoughtsAfter browsing for 14 hours with these programs, and recording all the results intospreadsheets, the most memory efficient browser in my usage is very clear--Firefox3.0 not only trumps its older version, but every other popular offering on Windows.This article may help other vendors rethink their marketing campaigns, and may promptfurther improvements.
[ 本帖最后由 Mallon 于 2008-6-25 18:01 编辑 ] |
发表于 2008-6-25 17:58
| 
