We currently have a number of users held back from updating due to websense related products detection. Unfortunately our first attempt at this used dll version information that wasn't accurate (the dll version we check doesn't get bumped with websense updates). So essentially we've blocked anyone with websense installed, even if they have a version of websense that is compatible with the latest revs of Firefox. We now have a new method of checking the current version of the websense product which we would like to use to update our gating. The version info is located in the registry: HKEY_LOCAL_MACHINE\SOFTWARE\Websense\Agent\InstallVersion The endpoint version format is major.minor.point (example: 8.2.2424) TESTING (prelim) SA1 - first rev of the system addon with the dll version checks SA2 - second rev developed here that check the registry version information - users who do not have websense installed successfully update from all major version starting at 47 to the latest after the SA2 installs - users who have an older endpoint (<=8.2.0) who have been held back by SA1 do not receive fx updates after updating to SA2 -- after these users update thier endpoint (>=8.2.5) with SA2 installed, fx updates on next update check - users who have the latest endpoint (>=8.2.5) and install firefox receive updates - users who have the latest endpoint (>=8.2.5) and install firefox with SA2 receive updates

I'm not 100% clear on where these websense users currently are in terms of fx version. I think it's 47.0.1, 48.0.3, and maybe 49.0. Ritu / Liz, do you all know the current location (version) of all of our held back websense users?

[Tracking Requested - why for this release]: Need to keep this bug on the release radar

(In reply to Jim Mathies [:jimm] from comment #0) > Unfortunately our first attempt at this used dll > version information that wasn't accurate (the dll version we check doesn't > get bumped with websense updates). So essentially we've blocked anyone with > websense installed, even if they have a version of websense that is > compatible with the latest revs of Firefox. The first version of the addon doesnt't check the version, but simply prevents all users with the qipcap.dll DLL from updating: https://hg.mozilla.org/releases/firefox-hotfixes/file/tip/v20160826.01/bootstrap.js#l32. The result is basically the same.

Jim, I think we could tell by looking at the logs for update pings. Rail or Ben, can you look at the logfiles you used before to see how many people & on which Firefox version we currently detect as having websense installed? I think checking all versions since 47.0 would be best (including the point releases).

Like Liz said Rail probably knows best here.

Note I have more detailed information on software versioning from the vendor if needed.

(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #4) > Jim, I think we could tell by looking at the logs for update pings. > > Rail or Ben, can you look at the logfiles you used before to see how many > people & on which Firefox version we currently detect as having websense > installed? I think checking all versions since 47.0 would be best > (including the point releases). This is best left to releaseduty folks. Rough instructions for how to do this are here: https://wiki.mozilla.org/Balrog#ELB_Logs

Track 51+ as we need to pay attention on the Websense issue.

Jim, you want this in a version prior to 51... correct? We shipped a one-off 47.0.2 with the websense Firefox system add-on which changes the app.update.url preference so it is possible for the server to know whether the system has websense installed. Also, 47.0.2 has the code to detect SSE2 and add it to the url so we only have to do one watershed. Unless I'm mistaken you want this in another one-off release which would be 47.0.3 or 48 but it would be better to include the SSE2 dtection code since if the system has websense it won't even startup with 49 and above. Does this sound right?

(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #9) > Jim, you want this in a version prior to 51... correct? We shipped a one-off > 47.0.2 with the websense Firefox system add-on which changes the > app.update.url preference so it is possible for the server to know whether > the system has websense installed. Also, 47.0.2 has the code to detect SSE2 > and add it to the url so we only have to do one watershed. Unless I'm > mistaken you want this in another one-off release which would be 47.0.3 or > 48 but it would be better to include the SSE2 dtection code since if the > system has websense it won't even startup with 49 and above. Does this sound > right? Yes this sounds correct. I think we also have users who are being held back on 48 as well, releng needs to get us numbers on where the websense users are so we can figure out what group(s) we want to target with a system addon update. 47 users are the main group as I under stand things, so we can go ahead and start working on those changes.

Gerry, regarding the "tracking-firefox51", this won't be shipping in Firefox 51 since if they have an affected version of websense installed then Firefox 51 won't start. What needs to be done here is a 47.0.3 release similar to the 47.0.2 release and we may choose to do the same on 48 as well.

Redirecting this to Jordan.

(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #11) > Gerry, regarding the "tracking-firefox51", this won't be shipping in Firefox > 51 since if they have an affected version of websense installed then Firefox > 51 won't start. What needs to be done here is a 47.0.3 release similar to > the 47.0.2 release and we may choose to do the same on 48 as well. Hi :rstrong, Thanks for reminder.

(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #4) > Jim, I think we could tell by looking at the logs for update pings. > > Rail or Ben, can you look at the logfiles you used before to see how many > people & on which Firefox version we currently detect as having websense > installed? I think checking all versions since 47.0 would be best > (including the point releases). Hi Liz, releaseduty folks are currently waiting on access for this. I've pinged in 1330651 to help expedite the process. In the mean time, has the request changed at all from the above description? Obviously the sooner the better but do we have a specific time this is needed by?

Jordan, no, not really. I think Jim would like to know for his talks with Websense folks. And I would like to know because I'm keeping an eye on the situation over time. Please let us know over email and not in the bug, though! Thanks.

ftr - just poking logs, we can only look at 47.0.2 users because it seems that's the only version we have a system-addon to detect websense. So we can only get that stats for that version specifically. I don't think we can provide info for every version from 47.0 onwards.

The system add-on should apply to all versions < 49. Manual testing indicates that the system add-on (Websense detection hotfix) also applies to versions 48.x. See regular update test results - https://docs.google.com/spreadsheets/d/1Hkz6q-LVUC6qKy0Ug9dFBFLYvz4__U2rhDtUOpywDus/edit#gid=1410574518. For 49.x+ the Websense hotfix installs and then immediately uninstalls, so we do no detection there. In practice, I know that we found out that there are some problems with applying the hotfix to everyone eligible (versions < 49), so only some people got it for some reason. I don't know if we've reached to any conclusions related to this yet.

We shipped the websense system add-on in 47.0.2 so the issue with pushing system add-ons to clients won't affect 47.0.2. If we ship an updated websense system add-on we will do the same as we did for 47.0.2 and include it in a build which would be 47.0.3 and possibly a 48.0.x release

[Tracking Requested - why for this release]: If we don't have this check in 51 (either in a dot release or in a system addon), WebSense users that are currently on 51 are going to crash when we release 52.

Marco, can you provide details of why this is the case? As I understood the previous websense bug we crashed on 48 and above so how would websense users not crash already on 48 and above?

WebSense breaks whenever we release a new Firefox build, as they use our internal symbols, but they release a new version shortly after we release (e.g. they recently released a version which is compatible with 51). So, when we release a new version, we should prevent people using an incompatible WebSense version from updating. For example, the reporter of bug 1335755 has a version which is compatible with 50 but not with 51, so he had to manually prevent Firefox from updating to 51.

(In reply to Marco Castelluccio [:marco] from comment #21) > WebSense breaks whenever we release a new Firefox build, as they use our > internal symbols, but they release a new version shortly after we release > (e.g. they recently released a version which is compatible with 51). My company machine has websense deployed and my firefox 52.0 beta installation is not usable anymore. I wonder why it is release-specific issue for firefox but not Chrome/IE. w.r.t. the KB on forcepoint at the time of writing, chrome 56 is not supported (yet) but it works while firefox 51/52 crashes. <https://support.forcepoint.com/KBArticle?id=TRITON-AP-ENDPOINT-DLP-Browser-Support-Matrix> I suppose corporate users usually have a current/beta version of firefox running before a compatible version forcepoint is deployed and crashes with firefox (my case). Alternatively, they will have machine image with forcepoint, download and install the latest release, and get the crashes. So eventually only those firefox ERS installations managed in corporate environment may survive this websense issue but not voluntary adopters who love firefox and enjoy the features under rapid release.

Ben / Jim, the additions to %OS_VERSION% will also include the values for detecting if the system is vulnerable to Bug 1296630 which was implemented in bug 1311515. Those possible values haven't changed and are: (unkBug1296630v1) (noBug1296630v1) (errBug1296630v1) (yesBug1296630v1) The values for websense will need to change to include the websense version. What I have is (nowebsense-null) = websense dll wasn't found and no registry version info (websense-null) = websense dll was found and no registry version info (nowebsense-*version*) = websense dll wasn't found and registry version info (websense-*version*) = websense dll was found and registry version info Note: *version* is in the format as noted in comment #0 Example without websense: /%OS_VERSION%(noBug1296630v1)(nowebsense-null)/ I am adding telemetry for these for the release so we can check what is going on with clients especially since this sounds like we're going to have this around for awhile. This sound good?

Felipe, since we are going to have to continue with the websense checks for the foreseeable future I went ahead and added histograms for the aushelper so it is easier to track. I could probably avoid some of the try catches but I'd rather be safe than sorry.

Comment on attachment 8834778 [details] [diff] [review] patch rev1 Benjamin, requesting data privacy review... I'd like telemetry for this since the only way we have to check the status currently is via the pings to balrog. Thanks!

> (nowebsense-null) = websense dll wasn't found and no registry version info > (websense-null) = websense dll was found and no registry version info > (nowebsense-*version*) = websense dll wasn't found and registry version info I'm not aware of this scenario, are you just covering all the bases? presumably if the version info is in the registry the software is installed. another check you could consider would be to see if their dll is loaded in our process space fi that's possible. > (websense-*version*) = websense dll was found and registry version info generally looks good to me.

(In reply to Jim Mathies [:jimm] from comment #26) > > (nowebsense-null) = websense dll wasn't found and no registry version info > > (websense-null) = websense dll was found and no registry version info > > (nowebsense-*version*) = websense dll wasn't found and registry version info > > I'm not aware of this scenario, are you just covering all the bases? > presumably if the version info is in the registry the software is installed. Just covering bases and that should be the case the vast majority of the time. Removing the additional checks would simplify things so let me know if you are ok with my removing them.

(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #23) > Ben / Jim, the additions to %OS_VERSION% will also include the values for > detecting if the system is vulnerable to Bug 1296630 which was implemented > in bug 1311515. > Those possible values haven't changed and are: > (unkBug1296630v1) > (noBug1296630v1) > (errBug1296630v1) > (yesBug1296630v1) > > The values for websense will need to change to include the websense version. > What I have is > (nowebsense-null) = websense dll wasn't found and no registry version info > (websense-null) = websense dll was found and no registry version info > (nowebsense-*version*) = websense dll wasn't found and registry version info > (websense-*version*) = websense dll was found and registry version info > > Note: *version* is in the format as noted in comment #0 > > Example without websense: > /%OS_VERSION%(noBug1296630v1)(nowebsense-null)/ > > I am adding telemetry for these for the release so we can check what is > going on with clients especially since this sounds like we're going to have > this around for awhile. > > This sound good? This should be fine for Balrog.

Comment on attachment 8834778 [details] [diff] [review] patch rev1 Can we do the pref setting outside of that big try/catch, so that we can also see WEBSENSE_REJECT_ERROR WEBSENSE_UNEXPECTED_ERROR? Also, please the the `+ "/"` next to the REPLACE_KEY like the previous code.. This was a trick we did to make sure this doesn't get replaced twice if the code runs twice. This was useful when the system-addon and hotfix shipped together, and is unlikely to be really needed here, but it's good to add this extra protection. For the telemetry probes, since some are opt-out, why not make the error code ones opt-out too? We try to not land expires=never probes anymore.. So what about setting it to version 60? If it's still needed we'll get an email alert on version 59 to consider bumping it again.

Comment on attachment 8835136 [details] [diff] [review] patch rev2 Jim, requesting review mainly to get your call on whether you are ok with not performing the file check. There should be few clients that either a) have the registry settings and don't have the files b) have the files but don't have the registry settings so it *should* be ok to remove the file check as long as you think that is ok. Thanks!

Comment on attachment 8835136 [details] [diff] [review] patch rev2 Benjamin, since these are new telemetry histograms requesting data steward approval. Thanks!

(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #31) > Comment on attachment 8835136 [details] [diff] [review] > patch rev2 > > Jim, requesting review mainly to get your call on whether you are ok with > not performing the file check. There should be few clients that either > a) have the registry settings and don't have the files > b) have the files but don't have the registry settings > > so it *should* be ok to remove the file check as long as you think that is > ok. > > Thanks! Also, felipe suggested we modify the url on unexpected failure. I'm not sure if this would be all that valuable for balrog since we already report this via telemetry and since this code will be included with the build any clients not having this in the url will be an unexpected failure anyways. I also don't expect the number of unexpected failure to be significant.

Comment on attachment 8835136 [details] [diff] [review] patch rev2 Felipe, I don't think we need to change the url for the failure cases since it would be reported to telemetry unless Jim thinks we should so I didn't add that. I'd like to get this reviewed before Jim's feedback since this will need to be uplifted to beta.

Comment on attachment 8835136 [details] [diff] [review] patch rev2 >diff --git a/toolkit/components/telemetry/Histograms.json b/toolkit/components/telemetry/Histograms.json >+ "AUSHELPER_WEBSENSE_REG_VERSION": { >+ "alert_emails": ["application-update-telemetry-alerts@mozilla.com"], >+ "bug_numbers": [1305847], >+ "expires_in_version": "60", >+ "kind": "boolean", >+ "keyed": true, >+ "releaseChannelCollection": "opt-out", >+ "description": "The Websense version from the registry." >+ }, This is strange. The expected way to record something like this is to use a scalar. Is that not possible for some reason having to do with uplift back to 42? This should at least record that the websense version is the "key" being used, because that's not clear. I'd like to see this again either with a scalar or a better description. And I didn't review the code, just the histograms.json file for data review. If there's anything else I need to review please let me know.

(In reply to Benjamin Smedberg [:bsmedberg] from comment #35) > Comment on attachment 8835136 [details] [diff] [review] > patch rev2 > > >diff --git a/toolkit/components/telemetry/Histograms.json b/toolkit/components/telemetry/Histograms.json > > >+ "AUSHELPER_WEBSENSE_REG_VERSION": { > >+ "alert_emails": ["application-update-telemetry-alerts@mozilla.com"], > >+ "bug_numbers": [1305847], > >+ "expires_in_version": "60", > >+ "kind": "boolean", > >+ "keyed": true, > >+ "releaseChannelCollection": "opt-out", > >+ "description": "The Websense version from the registry." > >+ }, > > This is strange. The expected way to record something like this is to use a > scalar. Is that not possible for some reason having to do with uplift back > to 42? This should at least record that the websense version is the "key" > being used, because that's not clear. I forgot scalars were now available. It appears that this will be the first time a 'kind: string' will be used outside of tests to accomplish this. Will try it out. > I'd like to see this again either with a scalar or a better description. And > I didn't review the code, just the histograms.json file for data review. If > there's anything else I need to review please let me know. Not a problem.

Changed to use a scalar (first 'kind: string' to be used in production) and I verified in about:telemetry that it recorded as expected.

Comment on attachment 8836243 [details] [diff] [review] patch rev 3 Carrying forward felipe's review.

Comment on attachment 8836504 [details] [diff] [review] patch rev 3 data-r=me

Comment on attachment 8836504 [details] [diff] [review] patch rev 3 Review of attachment 8836504 [details] [diff] [review]: ----------------------------------------------------------------- I think we can skip the file checks and just check the registry. r+ with that change.

Removed file checks, changed histogram name to reflect that it is a reg check, etc.

Comment on attachment 8838725 [details] [diff] [review] Patch rev4 Hi Felipe, see previous comment for changes made. They are significant enough that I think it would be a good thing to have you review it. I verified that telemetry shows up in about:telemetry and that the url is changed correctly locally.

Comment on attachment 8838725 [details] [diff] [review] Patch rev4 r=felipc over irc

Ben, here are a couple of examples of what the url pref will look like with the latest change. https://aus5.mozilla.org/update/6/%PRODUCT%/%VERSION%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%(noBug1296630v1)(nowebsense)/%SYSTEM_CAPABILITIES%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml https://aus5.mozilla.org/update/6/%PRODUCT%/%VERSION%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%(noBug1296630v1)(websense-8.2.2424)/%SYSTEM_CAPABILITIES%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml

Pushed by rstrong@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/088265acb42e Ship updated Firefox system add-on to change app.update.url preference for Websense with new endpoint version check. r=felipc, r=bsmedberg for data

Can we land this in 52 so that we will have a way to block updates from 52 to 53 for broken WebSense versions?

(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #45) > Ben, here are a couple of examples of what the url pref will look like with > the latest change. > > https://aus5.mozilla.org/update/6/%PRODUCT%/%VERSION%/%BUILD_ID%/ > %BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%(noBug1296630v1)(nowebsense)/ > %SYSTEM_CAPABILITIES%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml > > https://aus5.mozilla.org/update/6/%PRODUCT%/%VERSION%/%BUILD_ID%/ > %BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%(noBug1296630v1)(websense-8.2. > 2424)/%SYSTEM_CAPABILITIES%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml Thank you! That should be fine. Do we have a bug to track actually implementing more targetted blocks? Probably a question for Jim.

Comment on attachment 8838754 [details] [diff] [review] patch - landed Approval Request Comment [Feature/Bug causing the regression]: Websense crash - bug 1305847 [User impact if declined]: We won't be able to detect systems with Websense versions that don't / do crash and update them based on this. [Is this code covered by automated tests?]: No [Has the fix been verified in Nightly?]: I manually verified this in Nightly. [Needs manual test from QE? If yes, steps to reproduce]: Install websense and inspect the app.update.url preference value for addition of the (websense-X.X.X) value that is added. [List of other uplifts needed for the feature/fix]: None [Is the change risky?]: No [Why is the change risky/not risky?]: It is an additional registry check and update url preference modification performed by the AUSHelper system add-on similar to what it already does. [String changes made/needed]: None

Florin, can you help check if the patch is as expected?

Comment on attachment 8838754 [details] [diff] [review] patch - landed websense detection in aushelper, aurora53+, beta52+

Moving this to Alex for verification. I think we can check: - that the app.update.url pref contains the (websense-X.X.X) value only when Websense is installed - that the correct Websense value appears in the actual request - that update is performed only for Websense versions allowed - I guess we should be able to test this only when the rules are setup

I have checked the preff "app.update.url" before and after installing the Websense endpoint and can confirm that it contains the correct values: (nowebsense) when there is no endpoint installed and (websense-X.X.X) with the endpoint installed and running. The build I used was Nightly 54.0a1 (id: 20170221030205) which I've updated to the latest one (id: 20170222030329). The update was applied with no issues and did not affect the preff settings.

The Websense endpoint version that I used was 8.2.2324, the one that was causing the crashes.

Needs rebasing around bug 1320590 for the Beta uplift (or an uplift request in that bug too).

I requested uplift for bug 1320590

(In reply to Ben Hearsum (:bhearsum) from comment #50) > (In reply to Robert Strong [:rstrong] (use needinfo to contact me) from > comment #45) > > Ben, here are a couple of examples of what the url pref will look like with > > the latest change. > > > > https://aus5.mozilla.org/update/6/%PRODUCT%/%VERSION%/%BUILD_ID%/ > > %BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%(noBug1296630v1)(nowebsense)/ > > %SYSTEM_CAPABILITIES%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml > > > > https://aus5.mozilla.org/update/6/%PRODUCT%/%VERSION%/%BUILD_ID%/ > > %BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%(noBug1296630v1)(websense-8.2. > > 2424)/%SYSTEM_CAPABILITIES%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml > > Thank you! That should be fine. > > Do we have a bug to track actually implementing more targetted blocks? > Probably a question for Jim. Doesn't look like we do, but I spoke with Florin and Jim privately to try and figure out we want to do. Based on https://docs.google.com/spreadsheets/d/18n8NEimajNpnGRHdQsrrnyJ-Ufgvmpz-AXCUpVaFRMk/edit#gid=0, it seems to me that what we want to do is blacklist WebSense 8.2, which can be identified by looking for "(websense-8.2.23" in OS_VERSION. I've set-up a rule to do this on the release-cdntest channel. This means that: * WebSense users with SA1 (which sends "(websense)" in OS_VERSION) should continue to stay on 47.0.2 * WebSense users with SA2, and are running 8.2.23xx should continue to stay on 47.0.2 * WebSense users with SA2, and are running a *different* WebSense version (eg: 8.2.2424, 8.3.2509), will update to the latest Firefox. * Anybody without SA1 or SA2 (unable to determine WebSense status) will update to the latest Firefox. This probably needs some QA. And if anybody knows of *other* WebSense versions we should be blocking, please let me know. It's also worth noting that we must *not* ship the new System Addon until we have new blocking rules in place on the Firefox release channel.

I've ran a quick check trying to find other versions of Websense that would crash if 8.2 was the only version blocked. I have discovered that Websense version 8.0.1.2208 also makes Firefox crash, even in cases that we didn't expect (e.g. 47.0.1 x86, 47.0.2 x86, 48.0 x86 and others, on a Windows 10 x64 machine). What I think could be a good solution would be to blacklist all Websense versions smaller than 8.2 to avoid these crashes. Jim, do you think that this fix will reach Beta 9? If so we could copy the rule to beta-cdntest and also test if it's applied correctly. We cannot currently test this on release-cdntest.

(In reply to Alexandru Simonca, QA (:asimonca) from comment #61) > I've ran a quick check trying to find other versions of Websense that would > crash if 8.2 was the only version blocked. I have discovered that Websense > version 8.0.1.2208 also makes Firefox crash, even in cases that we didn't > expect (e.g. 47.0.1 x86, 47.0.2 x86, 48.0 x86 and others, on a Windows 10 > x64 machine). What I think could be a good solution would be to blacklist > all Websense versions smaller than 8.2 to avoid these crashes. Based on https://support.forcepoint.com/KBArticle?id=TRITON-AP-ENDPOINT-DLP-Browser-Support-Matrix, it looks like 8.0.1 never worked on Windows 10, so it seems sensible to blacklist it completely. However, that support matrix seems to say that 8.1 is compatible all the way up until Firefox 50. Are you sure we should be completely blacklisting 8.1? > Jim, do you think that this fix will reach Beta 9? If so we could copy the > rule to beta-cdntest and also test if it's applied correctly. We cannot > currently test this on release-cdntest. We haven't been doing any WebSense blacklisting on the beta channel (presumably because there's so few WebSense users on it). Why can't this be tested on release-cdntest? Is there anything I can do to make it possible?

> However, that support matrix seems to say that 8.1 is compatible all the way up until Firefox 50. Are you sure we should be completely blacklisting 8.1? Since we've found more start-up crashes on versions earlier than 8.2 I thought that this would be one solution for dealing with them but Jim should have the final word here. > We haven't been doing any WebSense blacklisting on the beta channel > (presumably because there's so few WebSense users on it). Why can't this be > tested on release-cdntest? Is there anything I can do to make it possible? We can't test this on release-cdntest because the only available build for testing it would be 52 RC but we wouldn't have anything to update it to. If you could copy the same rules from release-cdntest to beta-cdntest we would be able to use Beta 9 and update it to 52RC and test it.

(In reply to Alexandru Simonca, QA (:asimonca) from comment #63) > > However, that support matrix seems to say that 8.1 is compatible all the way up until Firefox 50. Are you sure we should be completely blacklisting 8.1? > > Since we've found more start-up crashes on versions earlier than 8.2 I > thought that this would be one solution for dealing with them but Jim should > have the final word here. I definitely think we should blacklist any versions that are crashing, I'm just not sure what the complete set of those are. > > We haven't been doing any WebSense blacklisting on the beta channel > > (presumably because there's so few WebSense users on it). Why can't this be > > tested on release-cdntest? Is there anything I can do to make it possible? > > We can't test this on release-cdntest because the only available build for > testing it would be 52 RC but we wouldn't have anything to update it to. If > you could copy the same rules from release-cdntest to beta-cdntest we would > be able to use Beta 9 and update it to 52RC and test it. You should be able to use older releases, and switch them to the release-cdntest channel. Eg: 47.0.2 might be a good choice, since that's the version we're holding certain WebSense users at. Feel free to find me on IRC if you have any trouble with this.

>You should be able to use older releases, and switch them to the release-cdntest channel. Eg: 47.0.2 might be a good choice, since that's the version we're holding certain WebSense users at. Feel free to find me on IRC if you have any trouble with this. That would be great but the older released versions of Fx don't contain the fix in this bug.

(In reply to Alexandru Simonca, QA (:asimonca) from comment #65) > >You should be able to use older releases, and switch them to the release-cdntest channel. Eg: 47.0.2 might be a good choice, since that's the version we're holding certain WebSense users at. Feel free to find me on IRC if you have any trouble with this. > > That would be great but the older released versions of Fx don't contain the > fix in this bug. Ah, because we haven't shipped the latest SystemAddon yet, I guess? Perhaps we should ship that on release-cdntest for now. The update changes I've made don't fully block websense users from updating, it just holds them at 47.0.2. I'm not sure it's possible to fully verify that without having the new SystemAddon installed on a 47.0.2 installation.

(In reply to Ryan VanderMeulen [:RyanVM] from comment #67) > https://hg.mozilla.org/releases/mozilla-beta/rev/fb4e4e5f4d69 And a bustage follow-up since record_in_processes isn't a thing on 52. https://hg.mozilla.org/releases/mozilla-beta/rev/6bff957177d2

(In reply to Alexandru Simonca, QA (:asimonca) from comment #61) > I've ran a quick check trying to find other versions of Websense that would > crash if 8.2 was the only version blocked. I have discovered that Websense > version 8.0.1.2208 also makes Firefox crash, even in cases that we didn't > expect (e.g. 47.0.1 x86, 47.0.2 x86, 48.0 x86 and others, on a Windows 10 > x64 machine). What I think could be a good solution would be to blacklist > all Websense versions smaller than 8.2 to avoid these crashes. > Jim, do you think that this fix will reach Beta 9? If so we could copy the > rule to beta-cdntest and also test if it's applied correctly. We cannot > currently test this on release-cdntest. Alexandru, could you update the matrix [1] with this information? [1] https://docs.google.com/spreadsheets/d/18n8NEimajNpnGRHdQsrrnyJ-Ufgvmpz-AXCUpVaFRMk/edit#gid=0 I think we should block anything that has a startup crash. The update should be delayed until the user or admin installs a newer version of websense.

https://hg.mozilla.org/releases/mozilla-esr52/rev/fb4e4e5f4d69 https://hg.mozilla.org/releases/mozilla-esr52/rev/6bff957177d2

I have updated the matrix with yesterday's testing. Everything that is marked with red is a startup crash. https://docs.google.com/spreadsheets/d/18n8NEimajNpnGRHdQsrrnyJ-Ufgvmpz-AXCUpVaFRMk/edit#gid=0