Closed
Bug 649155
Opened 14 years ago
Closed 13 years ago
Investigate FastestFox results
Categories
(Testing :: Talos, defect)
Testing
Talos
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: fligtar, Unassigned)
References
Details
Automated testing of FastestFox shows a start-up impact of 32% with the following data:
mysql> select * from perf_results where addon_id=9825;
+------+----------+---------------+--------------+---------+------+---------------------+---------------------+
| id | addon_id | appversion_id | osversion_id | average | test | created | modified |
+------+----------+---------------+--------------+---------+------+---------------------+---------------------+
| 429 | 9825 | 23 | 1 | 771.32 | ts | 2011-03-19 07:15:29 | 2011-03-19 07:15:29 |
| 401 | 9825 | 23 | 2 | 599.37 | ts | 2011-03-19 07:12:22 | 2011-03-19 07:12:22 |
| 705 | 9825 | 23 | 3 | 451.47 | ts | 2011-03-19 07:50:33 | 2011-03-19 07:50:33 |
| 471 | 9825 | 23 | 4 | 526.47 | ts | 2011-03-19 07:21:02 | 2011-03-19 07:21:02 |
| 1570 | 9825 | 24 | 1 | 819.47 | ts | 2011-04-09 07:26:17 | 2011-04-09 07:26:17 |
| 1448 | 9825 | 24 | 2 | 678.79 | ts | 2011-04-09 07:15:51 | 2011-04-09 07:15:51 |
| 1750 | 9825 | 24 | 3 | 566.26 | ts | 2011-04-09 07:49:26 | 2011-04-09 07:49:26 |
| 1535 | 9825 | 24 | 4 | 732.95 | ts | 2011-04-09 07:22:39 | 2011-04-09 07:22:39 |
+------+----------+---------------+--------------+---------+------+---------------------+---------------------+
8 rows in set (0.19 sec)
Manually testing start-up time we haven't been able to obtain similar results. We've removed the FastestFox results so that it doesn't appear on the slow performers page until we can figure out whether there's a problem in the automated testing.
Please investigate. Thanks.
Comment 1•13 years ago
|
||
I'm seeing the slow startup as reported by talos.
FastestFox is opening a second tab:
http://smarterfox.com/smarterwiki/installed/?from_ver=firstrun&to_ver=4.3.7
It brings up a 404. I'm not sure if it is just the overhead of the 404 affecting the results or if there is something else going on here.
Comment 2•13 years ago
|
||
I'm also seeing these console errors:
Could not read chrome manifest file '/Users/cltbld/talos-slave/talos-data/Firefox.app/Contents/MacOS/extensions/{972ce4c6-7e08-4474-a285-3208198ce6fd}/chrome.manifest'.
While creating services from category 'profile-after-change', service for entry '@smarterwiki/http-observer;1', contract ID '@smarterwiki/http-observer;1' does not implement nsIObserver.
Comment 3•13 years ago
|
||
Not sure where else to investigate here... FastestFox doesn't appear to be adding any unusual prefs to the browser - not sure if it attempting to preload something or other as that seems to be part of its modus operandi.
I think that being able to disable the first run tab would be a good start.
Comment 4•13 years ago
|
||
I ran some tests on my Mac OS machine and they also seem consistent with the results on the performance page. I also tried setting the preference that prevents the first run page from opening, but that didn't make a big difference in startup performance.
I'm also curious as to why the performance overhead on Mac is roughly half the one on Windows.
Alice, can you test on Windows with and without this preference set:
extensions.smarterwiki.last-version : 4.3.7
I'm interested in knowing if this makes a major difference in the measured impact.
Comment 5•13 years ago
|
||
I'm trying to get those tests run, but having some difficulty with our staging boxes. I'll get the data posted as quickly as I can gather it.
Comment 6•13 years ago
|
||
Using nightly firefox-4.0.2pre.en-US.win32.zip (pulled today) on winxp:
without pref:
i,val
0,712
1,539
2,590
3,576
4,522
5,526
6,592
7,542
8,544
9,516
10,536
11,516
12,544
13,672
14,574
15,537
16,520
17,531
18,537
19,525
Average excluding max: ts: 549.42
With pref:
i,val
0,685
1,532
2,516
3,523
4,678
5,527
6,541
7,551
8,519
9,577
10,531
11,556
12,525
13,511
14,542
15,537
16,592
17,539
18,526
19,512
Average excluding max: ts-pref: 543.95
Any help here? I can run the tests against another build if you need more info.
Comment 7•13 years ago
|
||
Aren't these averages much lower than those on comment #0, and more in line with the results in other systems?
Comment 8•13 years ago
|
||
It could be in line with the results in comment #0, I'm just not sure what os_version maps to which os.
Comment 9•13 years ago
|
||
mysql> select id,os,version,name from perf_osversions;
+----+--------+------------------+-----------------+
| id | os | version | name |
+----+--------+------------------+-----------------+
| 1 | MacOSX | 10.5.8 | Mac OS X 10.5.8 |
| 2 | Fedora | 12 - Constantine | Fedora 12 |
| 3 | WINNT | 5.1 | Windows XP |
| 4 | WINNT | 6.1 | Windows 7 |
+----+--------+------------------+-----------------+
4 rows in set (0.00 sec)
mysql> select id,app,version from perf_appversions;
+----+------+-----------------+
| id | app | version |
+----+------+-----------------+
| 1 | | browser_version |
| 2 | | browser_version |
| 3 | | browser_version |
| 4 | | browser_version |
| 5 | | browser_version |
| 6 | | browser_version |
| 7 | | browser_version |
| 8 | | browser_version |
| 9 | | browser_version |
| 10 | | browser_version |
| 11 | | browser_version |
| 12 | | browser_version |
| 13 | | browser_version |
| 14 | | browser_version |
| 15 | | browser_version |
| 16 | | browser_version |
| 17 | | browser_version |
| 18 | | browser_version |
| 19 | | browser_version |
| 20 | | browser_version |
| 21 | fx | 3.6.15pre |
| 22 | fx | 4.0b13pre |
| 23 | fx | 3.6.15 |
| 24 | fx | 4.0 |
| 25 | fx | 4.0.1 |
+----+------+-----------------+
25 rows in set (0.00 sec)
Comment 10•13 years ago
|
||
Thanks, Wil. I assumed XP was 1, so I was wrong in Comment #7.
It also looks like the difference with and without the preference on Windows is not that big, so it wouldn't be worth including it in the default configuration.
I don't know what else we can do here. We should probably close this bug since it appears that Talos is doing the right thing.
Comment 11•13 years ago
|
||
Based upon comment 10, closing.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•