As discussed in Portland, we'll want to add telemetry to analyze the effectiveness of the patch in bug 1272585 in reducing the number of users on old versions of Firefox, in particular users stuck in the CHK_IS_DOWNLOADING state. This bug is to implement and analyze the situation before and after landing bug 1272585 and will therefore need to land before bug 1272585.

Existing telemetry might be enough to do the analysis, but this bug would serve as the right place to track code changes if any are necessary.

Data for the analysis of the Firefox Beta channel for 48 and 49. The query was limited to 2 million records. Total number of records with a download status code per Beta version Beta 48 = 25634 Beta 49 = 30499 The first column is the count of the times a client sent 12 (e.g. NS_ERROR_DOCUMENT_NOT_CACHED) as the download status. The other columns are the number of clients that have sent 12 for the download status code for the count in the first column. So, if a client was running Beta 49 and sent 12 as the download status 3 times for a complete update they would be included under Complete Update in the 49 column and the row represented by 3 in the Reports column. If the same client sent 12 as the download status 4 times for a partial update they would also be included under Partial Update in the 49 column and the row represented by 4 in the Reports column. | Complete Update | Partial Update | Reports | 48 | 49 | 48 | 49 | --------+--------+--------+--------+--------+ 1 | 3919 | 1627 | 2629 | 294 | 2 | 2059 | 584 | 845 | 98 | 3 | 1222 | 291 | 358 | 33 | 4 | 776 | 157 | 189 | 18 | 5 | 569 | 97 | 112 | 4 | 6 | 409 | 66 | 59 | 7 | 7 | 304 | 36 | 38 | 2 | 8 | 235 | 22 | 43 | 3 | 9 | 175 | 19 | 16 | 2 | 10 | 134 | 8 | 12 | 1 | 11+ | 884 | 54 | 44 | 1 | --------+--------+--------+--------+--------+ Total | 10686 | 2961 | 4345 | 463 | The percentage of records with a download status of 12 (e.g. NS_ERROR_DOCUMENT_NOT_CACHED) out of the total number of records with a download status code. For reference once again Total number of records with a download status code per Beta version Beta 48 = 25634 Beta 49 = 30499 | Complete Update | Partial Update | Reports | 48 | 49 | 48 | 49 | --------+--------+--------+--------+--------+ 1 | 15.29% | 5.33% | 10.26% | 0.96% | 2 | 8.03% | 1.91% | 3.30% | 0.32% | 3 | 4.77% | 0.95% | 1.40% | 0.11% | 4 | 3.03% | 0.51% | 0.74% | 0.06% | 5 | 2.22% | 0.32% | 0.44% | 0.01% | 6 | 1.60% | 0.22% | 0.23% | 0.02% | 7 | 1.19% | 0.12% | 0.15% | 0.01% | 8 | 0.92% | 0.07% | 0.17% | 0.01% | 9 | 0.68% | 0.06% | 0.06% | 0.01% | 10 | 0.52% | 0.03% | 0.05% | 0.00% | 11+ | 3.45% | 0.18% | 0.17% | 0.00% | --------+--------+--------+--------+--------+ Total | 41.69% | 9.71% | 16.95% | 1.52% |

Since a complete takes longer than a partial to download it makes sense that the reduction of these errors is greater for the partial now that the download continues from where it left off instead of starting the download over.

Bug 1272585 made it so when the NS_ERROR_DOCUMENT_NOT_CACHED error occurs instead of starting the download over it continues from where the download was when the error occurred. The data above shows that the error is reported significantly less often and that the reduction is much greater with partial update downloads both of which were expected. As stated in Portland, since the download starts over from the beginning the only thing needed for the update to succeed is for Firefox to not have this error when downloading which could be accomplished by running long enough for the download to complete. Though we could spend time (which would be a significant amount of time) diving into the data further the update orphaning dashboard should show if there is any affect on update orphaning when 49 is a couple of releases behind the current release.

Followup: For the first time since the dashboard was implemented this result code (12 DWNLD_ERR_DOCUMENT_NOT_CACHED) was not the largest result code on the dashboard's "Out of date, of concern client last update download code (download phase)" section and the new largest result code is 0 DWNLD_SUCCESS. This confirms that the work in bug 1272585 had a significant positive effect.