Closed
Bug 648229
Opened 14 years ago
Closed 13 years ago
Compatibility info of the add-ons isn't properly considered during performance testing
Categories
(Testing :: Talos, defect)
Testing
Talos
Tracking
(Not tracked)
RESOLVED
WONTFIX
People
(Reporter: ecfbugzilla, Assigned: anodelman)
References
Details
Attachments
(1 file)
(deleted),
patch
|
jmaher
:
review+
|
Details | Diff | Splinter Review |
I tried to understand why Adblock Plus got a perfect score in the March 26th performance testing run. Turns out that this run tested Adblock Plus 1.3.3 because Adblock Plus 1.3.5 wasn't reviewed yet. And Adblock Plus 1.3.3 has an install.rdf file saying that it is only compatible with Firefox 4.0b9pre. It was marked as compatible to the Firefox 4 release on addons.mozilla.org but that info wasn't considered - so the performance test checked a disabled add-on and gave it perfect marks.
Talos should test extension compatibility *before* downloading an add-on, incompatible add-ons shouldn't even be downloaded. And it should make sure a compatible add-on isn't disabled because of install.rdf info, probably easiest by setting checkCompatibility pref to false.
Updated•14 years ago
|
Blocks: AddonSlowStartup
Assignee | ||
Comment 1•14 years ago
|
||
If we download an incompatible addon there should be a compatibility check pre-testing... there is no point in testing an addon that a user will be unable to installer in their particular browser.
I believe that a graceful fail would be sufficient here by comparing the addon information to the browser.ini info.
Assignee: nobody → anodelman
Reporter | ||
Comment 2•14 years ago
|
||
To clarify: there are two sources of compatibility info - the extension itself and addons.mozilla.org info. The latter can be updated without updating the extension, particularly when a new browser version comes out. Because of this 30 add-ons tested by Talos are currently not marked as compatible with Firefox 4 in the extension - but most of them are marked as compatible on addons.mozilla.org. When Talos installs add-ons it doesn't give the browser a chance to query addons.mozilla.org for current compatibility info. So an extension that would be compatible isn't tested.
I think that setting extensions.checkCompatibility.4.0 pref to false would be the best short-term solution - the add-ons that are really not compatible with Firefox 4 are very few. For the final solution it is probably possible to get the list of add-ons generated dynamically on AMO so that incompatible add-ons are not even included. Otherwise the only way to get current compatibility info is to parse https://services.addons.mozilla.org/en-US/firefox/api/1/addon/1865.
Reporter | ||
Comment 3•14 years ago
|
||
(In reply to comment #2)
> So an extension that would be compatible isn't tested.
To be precise: it is tested but the browser disables it as incompatible.
Assignee | ||
Comment 4•14 years ago
|
||
If skipping the compatibility check is reasonable I'm willing to do it - I only fear testing addons that realistically are not compatible with a given version of the browser...
Comment 5•14 years ago
|
||
Incompatible add-ons should be rare, specially within the testing set we're currently using.
Reporter | ||
Comment 6•14 years ago
|
||
Right now, occasionally testing an incompatible add-on should be better than testing 30 compatible add-ons in a disabled state. But of course this needs to be fixed properly later.
Assignee | ||
Comment 7•14 years ago
|
||
What is the pref that needs setting?
Reporter | ||
Comment 8•14 years ago
|
||
extensions.checkCompatibility.4.0 : false
Assignee | ||
Comment 9•14 years ago
|
||
Attachment #526844 -
Flags: review?(jmaher)
Comment 10•14 years ago
|
||
Comment on attachment 526844 [details] [diff] [review]
[checked in]disable compatability check for 4.0
so simple, what can I find to nit with this patch?
Attachment #526844 -
Flags: review?(jmaher) → review+
Assignee | ||
Comment 11•13 years ago
|
||
Comment on attachment 526844 [details] [diff] [review]
[checked in]disable compatability check for 4.0
changeset: 232:ca836ce72df2
Attachment #526844 -
Attachment description: disable compatability check for 4.0 → [checked in]disable compatability check for 4.0
Assignee | ||
Comment 12•13 years ago
|
||
Deployed.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
Reporter | ||
Comment 13•13 years ago
|
||
Reopening - what got deployed here is only a quick work-around. Compatibility info is still not being considered, incompatible add-ons could be tested.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee | ||
Comment 14•13 years ago
|
||
As a note, this will be fixed as a side effect of the new on-demand addon testing system - as only the correct addon for a given os will be pushed into the system for testing.
Updated•13 years ago
|
Status: REOPENED → RESOLVED
Closed: 13 years ago → 13 years ago
Resolution: --- → WONTFIX
You need to log in
before you can comment on or make changes to this bug.
Description
•