User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:39.0) Gecko/20100101 Firefox/39.0 Build ID: 20150630154324 Steps to reproduce: Update Firefox from 39.0 to 40.0b1 Actual results: Crashes on startup, even with a new profile. Tried both x86 and x64 builds without difference. See crash reports: bp-2183b0c6-8799-4149-9938-727172150707 bp-9850ebf6-f1b5-4d0d-9426-4a95b2150707 bp-68abd9da-b8af-4a59-ab33-04b852150707 bp-5cb15dcc-09f2-479d-b077-509922150707 bp-9a620b14-e92e-4c5d-83bf-0be752150706 bp-28c76252-581c-44f4-b7ed-f859c2150706 bp-bfd83a06-db10-43a3-9351-68da32150706 bp-d308adf3-b00d-4ebb-bad8-b7e002150706 bp-aa89180b-6a5b-4f10-aa1c-e9e032150706 Expected results: Launch normally.

If you clean-install in a separate directory (I imagine we'd want to keep the old broken copy to figure out what went wrong), do you see the same thing? Benjamin, any idea what's going on here?

There is a semi-interesting DLL AppHookWIN6032_075EAC1C-397B-42F9-9E72-8D68DBC8F427.dll in these crash reports. There is also sometimes a Fingertype.dll So I strongly suspect malware/virus activity here. Sander, in your Firefox install directory, can you please look for a directory "distribution" and if there is anything there, send me an archive of the contents? The default Firefox install directory is "C:\Program Files (x86)\Mozilla Firefox"

I will take a look tomorrow - but have to add that the issue does not occur anymore after (a) updating AMD drivers to a 15.x beta version (from 12.x release) for the Radeon HD 5450 _and_ installing Beta 2 of FF 4.0. Sorry that I made two changes at once.

- No directory "distribution" exists. - A scan with Malwarebytes does not reveal any malware. Both systems are corporate desktops with McAfee AV and a scanning proxy server in front. - Firefox 40 b1 also starts successfully, with the only change the updated video driver. - On both systems, DisplayFusion 7.2b8 is installed for multimonitor support, which hooks into programs. This explains the AppHook*.dll files. - Fingertype.dll comes from Fingertips (getfingertips.com), a keyboard utility/launcher. I believe we can conclude that the issue has been caused by a change in Firefox 40 with regards to video drivers, which might be incompatible with some. Compatible: AMD-Catalyst-15.6-Beta-Software-Suite-Win7-64Bit-June22 (beta driver) Incompatible: AMD-Catalyst-Omega-14.12-With-DOTNet45-Win7-64bit (latest production version currently)

The stack contains "QIPCAP.dll" which is part of "Websense Endpoint". All reports of this crash have version 7.6.818.1 of that file. Sander: if you still experience this crash, could you check to see if there's a newer version of Websense available?

[Tracking Requested - why for this release]: Startup crash, about 5% of v40 crashes in early data

apparently this crash is occurring with the latest version of "Websense Endpoint" (8.0.2076) that's available. https://support.mozilla.org/en-US/questions/1077273

Some crash ID reported by the user of the duplicate bug: bp-bbf9a449-f74e-4497-b617-1050b2150812 bp-afeec441-45c1-4239-8d40-8b7462150812 bp-f82b1d30-97f5-4fae-8d76-d57232150812 bp-3eca6e86-a18d-46fe-86cc-d6c1f2150812 He has Kasperky Endpoint Security 10 installed too.

I'm the reporter of bug 1193713 The version of Kasperky Endpoint Security 10 installed on my system is 10.2.2.10535

(Copying from the other bug) The suspected software is called Websense Endpoint. I believe that's different from Kaspersky Endpoint.

See also bug 1190437 also blaming a websense update.

Crashes are bad, tracked.

We have another report for this in the support forums: https://support.mozilla.org/en-US/questions/1077927

For tracking purposes, we have another report here: https://support.mozilla.org/en-US/questions/1077847

This affects me too and I reported here 1193653 (duplicated). That happened on my corporate PC with McAfee AV solution and TRITON AV Endpoint.Is there any temporary solution because it severely affects my Sync based bookmark and other usage.

(In reply to David Major [:dmajor] from comment #5) > The stack contains "QIPCAP.dll" which is part of "Websense Endpoint". All > reports of this crash have version 7.6.818.1 of that file. > > Sander: if you still experience this crash, could you check to see if > there's a newer version of Websense available? See also bug 828184. The mentioned file QIPCAP.DLL comes with Websense Endpoint (1.5.)8.0.2076 which is the current version. We do have McAfee configured to not scan Websense files. See also: https://www.websense.com/support/article/kbarticle/exclude-endpoints-from-scanning Currently I do not experience this crash myself (see comment 4), however numerous others seem to have the issue.

Edit: on the PCs of my users, Firefox 40.0.2 crashes on startup right away while Firefox 39 did not. bp-81d8039d-5f15-4949-8cb2-dc1282150817 bp-867083dd-1b6f-49d7-adba-01ef42150817

Sylvestre: Can you reach out to the vendors of this software for help? Florin: Can your team try to find an installer that reproduces this issue? Bob: Here's another AV crash Did something change in 40 regarding sandboxing, etc.?

Sylvestre, if necessary I can open a support call with Websense, it will go through support to development so if you have a direct contact that would better - otherwise just contact me.

(In reply to David Major [:dmajor] from comment #20) > Bob: Here's another AV crash Did something change in 40 regarding > sandboxing, etc.? There was a slight sandboxing policy change for GMP, but not something that I think could have caused a crash. Also, this doesn't appear to be GMP related, no other child processes are sandboxed in 40.

(In reply to David Major [:dmajor] from comment #20) > Florin: Can your team try to find an installer that reproduces this issue? This seems pretty complicated to setup. I've registered on their site and apparently they will contact me to help me install a trial version of Web Security, which may then allow me to install a Websense Endpoint... I'm not very optimistic about this.

So still no news from Websense, but Sander sent me an installer and I was able to easily reproduce the issue just by installing the endpoint as specified at http://www.websense.com/content/support/library/shared/v78/endpoints/web_endpoint.pdf (msiexec /package "Websense Endpoint.msi" /norestart WSCONTEXT=xxxx) and then started Firefox. Firefox 39.0.3 starts without problems. Firefox 40.0.2 crashes at startup - bp-023b5392-7185-40b3-8786-d0a5a2150818.

We don't have any contact afaik. Sander, it would be great if you could contact them. (or, if you give me an email address, I can contact them)

Sylvestre: already sorted, see comment #24.

Sander, Hmm, I don't think so. I was asking for a contact from upstream to let them know about the bug and work with them. (however, having a way to reproduce it is great, thanks!)

Sylvestre, I will try to get a 'warm' contact for you. Their normal support mailbox is support@websense.com which you might already have tried.

I wonder if this is related to the mozalloc removal? Perhaps qipcap.dll is calling some free-like function in response to this ScopedXPCOM callback. Glandium, is there anything Florin or Sander can do to test this theory?

Sylvestre: Case 02221686 has been raised with the vendor for this issue, with reference to this bug. They have been requested to get in touch.

Update from Websense: == The last version of firefox that is currently supported is version 39.x. Our development teams are currently working on an Endpoint version for versions 40.x . The browser versions currently supported are provided in the URL below. http://www.websense.com/content/endpointproductmatrix.aspx I suggest watching for tech alerts on the new EP release and always cross checking the above matrix ==

That's quite unfortunate. Crashing the browser on startup is a pretty harsh way of saying that the configuration is not supported, especially when the end user has no indication that the crash is due to Websense.

Attaching the system32\qipcap.dll as instructed by David. Unfortunatelly I haven't been able to get any minidumps yet.

I've managed to get a minidump with WinDBG and sent it to David.

Thanks for the memory dump, Florin. It looks like Websense is hooking NS_GetServiceManager, and their hook tries to jump through a function pointer table entry that hasn't been populated (maybe some FF export that disappeared in 40?). Could you try a few more things for me: - Collect a full-memory dump from a working installation on 39 (use CrashMe, if necessary) - Back on version 40, try replacing mozglue.dll with the attached version. This will test whether our DLL blocklist can stop this software.

Oops, try this one instead. Apparently we already blocked qipcap.dll up to version 7.6.815.1 in bug 828184. This expands the block to 7.6.818.1.

(In reply to David Major [:dmajor] from comment #35) > - Collect a full-memory dump from a working installation on 39 (use CrashMe, > if necessary) Collected and sent via email. > - Back on version 40, try replacing mozglue.dll with the attached version. > This will test whether our DLL blocklist can stop this software. 40.0.2 still crashes on startup. Collected a minidump and sent via email.

From Florin's minidump, on version 39 Websense still has some null pointers in that function table, but it doesn't become a problem because they don't hook NS_GetServiceManager in those builds. Unclear why the difference. Possibly some mistaken function locations due to SDK changes. The fact that the blocklist experiment in test_mozglue.dll didn't work, is troubling. It means that we can't do anything to mitigate this from the Firefox side.

Looks like we won't be able to take a workaround in 40. Hopefully, they will fix that in their software.

This not high volume crash in FF41. Resolving as wontfix. Given that it is not actionable on our side from the previous two comments, I am not adding a tracking flag for 42. If this spikes up, we can try to determine some next steps outside of a Firefox fix.

I can confirm i am seeing consistent crashes on FF startup on a handful of machines running FF 40.0.3 AND Kaspersky Endpoint 10 (10.2.1.23 a) The crash only occurs on first boot of the PC. You get the "well this is embarassing..." screen and then it works fine on subsequent close/reopens, until next reboot. Was able to reproduce consistently and by disabling the AV before first firefox load after a reboot/boot, the issue is no longer there!

New Websense Triton-AP Endpoint client 8.0.2208 causes crashes after version 36 of Firefox. Previous Endpoint Client worked fine with all versions of Firefox. Once uninstalled the Websense client and reboot, Firefox works on all versions.

(In reply to EJW from comment #43) > New Websense Triton-AP Endpoint client 8.0.2208 causes crashes after version > 36 of Firefox. That version might have been pulled, I only find version 8.0.2076 currently which still officially supports Firefox up to version 36. With the new version you could try to exclude firefox.exe from interception (Web > Endpoint > Endpoint Bypass) - it doesn't seem to set the proxy settings correctly anyway.

(In reply to EJW from comment #43) > New Websense Triton-AP Endpoint client 8.0.2208 causes crashes after version > 36 of Firefox. I now see the version number is 8.1.0.2208. Release notes here: https://www.websense.com/content/support/library/endpoint/v801a/release_notes/rn_endpt_new.aspx#705898 Support has been added for Firefox up to v39 depending on the version. Full matrix here: http://www.websense.com/content/SupportedProductMatrix.aspx

Websense Endpoint 8.0.1.2208 / 8.0.2208 still contains QIPCAP.DLL version 7.6.818.1. Firefox 40.0.3 still crashes: bp-4160ff92-05a4-41fb-ad68-99f142150907 Making a process exception for firefox.exe does not make a difference. Interestingly Firefox x64 runs fine with QIPCAP64.DLL which has the same version number as the 32 bit DLL.

Sylvestre, did you ever get through to someone at Websense? Could you try pinging again please?

Reverting back to Websense Client 8.0.2048/Policy Engine Version 8.0.0.153 seems to solve the issue as a temporary fix.

I didn't as Sander Goudswaard contacted them. Sander, could you give me the email address of your contact? Thanks

(In reply to Sylvestre Ledru [:sylvestre] from comment #49) > I didn't as Sander Goudswaard contacted them. > Sander, could you give me the email address of your contact? Thanks I only have the support@websense.com e-mail address and the case number 02221686. My contact goes through a company that handles our network security. I could ping them again, however since the official statement from Websense is that they don't support Firefox 40 yet - that is not likely to succeed. I will see if I can find an individual who understands the need for Firefox to run correctly with Websense Endpoint.

I have been notified that Websense has resolved the issue in Endpoint version 8.0.2224 and will test this on Monday.

Websense is still testing an Endpoint client that contains a fix for the issue. Once I have more information, I will update this bug.

Thanks again! Seems like I won't need to contact them. If you are in touch with them, please ask them to test against 41: it will be released next week.

The QA team has tested with Firefox 41 and found issues with HTTP POST that have been resolved in Websense Endpoint versions 8.1.2228+ (no ETA).

Today, Websense released endpoint version 8.1.2224 which should support both Firefox 40 and Windows 10. I cannot confirm this, since Firefox 40 (x86 only) still crashes. bp-8ede0220-b57f-427f-a10d-d11a42150930

I confirm the new version from Websense endpoint 8.1.2224 have fixed the start up issue with Firefox, updated to 41.0.1

Confirmed. After installation of the new Websense Endpoint version, the computer may need a restart for the changes to take effect. The Websense Endpoint may cause future versions of Mozilla Firefox to malfunction. Such issues will need to be directed to Websense Support. For organizations, the ESR versions of Firefox are less likely to cause issues with Websense.

FYI for anyone else hopping by here, I'm running Websense Triton AP version 8.1 build 2224 and Firefox 42 and am still having the crash problems. bp-7fef592d-47c6-44cc-b749-30a162151123 I uninstalled it and FF opened right up. I'm working with our Security guy at work to see if he can contact Websense to have someone look at this.

Hi all Also had the same issue where dev's had to test sites using FF 43, but the Websense 8.1 build 2224 still made it crash. Unfortunately we cant uninstall websense due to business policy so I figured out a quick hack to get you going. Fired up Immunity Debugger and saw in firefox.exe that the QIPCAP.dll was making it crash. So using reflection its possible to bypass this issue. 1. Open text editor(notepad) as administrator. 2. Save the empty file as "QIPCAP.dll" within your Firefox install directory Default: "C:\Program Files (x86)\Mozilla Firefox files(x86)" 3. Start Firefox -- A dll load error will pop up. Just click ok and Firefox will start fine. To check where the dll is currently loaded, run depending on OS architecture: tasklist /FI "MODULES eq qipcap.dll" tasklist /FI "MODULES eq qipcap64.dll" Hope this helps as a quick hack until websense figure out their bugs.

This issue is because Websense only makes their products compatible with the ESR version of Firefox and not the normal releases. All you have to do is installed Firefox ESR instead of regular Firefox and everything will work fine.

The next ESR is 45, so I hope they are working on it. :)

Thanks for the workaround Jurgens! To get around the error message about the invalid DLL, I created an empty one (well as empty as I could make it in a short time). If you place this DLL in `c:\Program Files (x86)\Mozilla Firefox` then Firefox starts with no DLL errors and doesn't crash.

Hello all, thank you for your efforts to collaborate. I want to provide information for direct contact regarding all product security issues in all Forcepoint products and services: I am a founding member of the Forcepoint Global Product Security Incident Response Team (PSIRT). (Forcepoint is the synthesis of Raytheon Cyber Products, Websense, Trusted Computer Solutions, Oakley Networks, Visual Analytics, Stonesoft, Sidewinder, and others) If you ever need to report a product security vulnerability, contact PSIRT@forcepoint.com. PGP Key: https://www.forcepoint.com/innovation/product-security/product-security-report-issue