It's #5 top browser crasher in the first hours of 18.0. It's a startup crash but with not so many duplicates. QIPCAP.dll (version 7.6) is a DLL that belongs to Websense Endpoint (https://www.websense.com/content/Home.aspx). Signature nsPrefBranch::GetIntPref(char const*, int*) More Reports Search UUID de4a2771-9d31-474a-b784-df66f2130109 Date Processed 2013-01-09 09:22:19 Uptime 0 Last Crash 26.5 minutes before submission Install Age 26.9 minutes since version was first installed. Install Time 2013-01-09 08:54:58 Product Firefox Version 18.0 Build ID 20130104151925 Release Channel release OS Windows NT OS Version 6.1.7601 Service Pack 1 Build Architecture x86 Build Architecture Info GenuineIntel family 6 model 23 stepping 10 Crash Reason EXCEPTION_ACCESS_VIOLATION_WRITE Crash Address 0x2 App Notes AdapterVendorID: 0x8086, AdapterDeviceID: 0x2a42, AdapterSubsysID: 024d1028, AdapterDriverVersion: 8.15.10.2302 EMCheckCompatibility True Adapter Vendor ID 0x8086 Adapter Device ID 0x2a42 Total Virtual Memory 2147352576 Available Virtual Memory 1933312000 System Memory Use Percentage 61 Available Page File 3456798720 Available Physical Memory 1414131712 Frame Module Signature Source 0 xul.dll nsPrefBranch::GetIntPref modules/libpref/src/nsPrefBranch.cpp:181 1 QIPCAP.dll QIPCAP.dll@0x1e840 2 QIPCAP.dll QIPCAP.dll@0x1f726 3 QIPCAP.dll QIPCAP.dll@0x214e4 4 xul.dll nsObserverService::NotifyObservers xpcom/ds/nsObserverService.cpp:149 5 xul.dll nsCOMPtr_base::~nsCOMPtr_base obj-firefox/dist/include/nsCOMPtr.h:408 6 mozalloc.dll mozalloc.dll@0x109f 7 @0xd 8 mozglue.dll je_free memory/mozjemalloc/jemalloc.c:6589 9 xul.dll nsLocalFile::Release xpcom/io/nsLocalFileWin.cpp:977 10 xul.dll nsCOMPtr_base::~nsCOMPtr_base obj-firefox/dist/include/nsCOMPtr.h:408 11 xul.dll NS_InitXPCOM2_P xpcom/build/nsXPComInit.cpp:447 12 xul.dll ScopedXPCOMStartup::Initialize toolkit/xre/nsAppRunner.cpp:1181 13 xul.dll XRE_main toolkit/xre/nsAppRunner.cpp:3935 14 firefox.exe wmain toolkit/xre/nsWindowsWMain.cpp:105 15 firefox.exe __tmainCRTStartup crtexe.c:552 16 ntdll.dll __RtlUserThreadStart 17 crypt32.dll ErrLog_LogError 18 kernel32.dll WerpReportFaultInternal 19 kernel32.dll WerpReportFaultInternal More reports at: https://crash-stats.mozilla.com/report/list?signature=nsPrefBranch%3A%3AGetIntPref%28char+const*%2C+int*%29

I've attempted to contact Websense about this.

CCing bsmedberg here to check if this could be related to https://bugzilla.mozilla.org/show_bug.cgi?id=828296 & recommendation regarding going ahead and blocklisting here.

Yes, this is very likely related to bug 828296, and the extension needs to be recompiled to take that into account.

I don't see any extensions in the crash report. Maybe this is a DLL being injected into Firefox in some other way?

Indeed. But it's still using XPCOM APIs after it manages to get itself injected.

This is starting to appear on SUMO. Any word from websense?

No response yet. I'm adding the only contact on Bugzilla that seems to still be valid. Note that this would need to be an in-product DLL block if we chose to go that way.

(In reply to Jorge Villalobos [:jorgev] from comment #7) > No response yet. I'm adding the only contact on Bugzilla that seems to still > be valid. > > Note that this would need to be an in-product DLL block if we chose to go > that way. I'm also trying to give WebSense a call right now. I want to verify that they either plan to fix the issue in the short term for 7.6 users, or what the user experience will be like if we block the DLL. (In reply to Tyler Downer [:Tyler] from comment #6) > This is starting to appear on SUMO. For reference: https://support.mozilla.org/en-US/questions/946986?esab=a&s=websense&r=1&as=s And crash comments: https://crash-stats.mozilla.com/report/list?range_value=7&range_unit=days&date=2013-01-14&signature=nsPrefBranch%3A%3AGetIntPref%28char%20const*%2C%20int*%29&version=Firefox%3A18.0#comments

No dice on the phone. Somebody will call me back, but the technical support people need to be contacted by customers. We need to grab some emails to perform outreach, so that their IT can contact Websense. Is there any evidence that v7.6 of QIPCAP.dll is working for some users? If not, we should go ahead and blocklist.

Marcia helped with the emails, since she's PT.

Note that the block would have to be in-product, given that this is a DLL block.

(In reply to Alex Keybl [:akeybl] from comment #9) > Is there any evidence that v7.6 of QIPCAP.dll is working for some users? I have no idea how I should find out about that.

It happens with recent QIPCAP.dll versions, including the latest one: nsPrefBranch::GetIntPref(char const*, int*)|EXCEPTION_ACCESS_VIOLATION_WRITE (347 crashes) 100% (347/347) vs. 0% (352/152000) QIPCAP.dll 0% (0/347) vs. 0% (1/152000) 7.6.808.1 0% (0/347) vs. 0% (1/152000) 7.6.811.1 3% (9/347) vs. 0% (11/152000) 7.6.812.1 3% (10/347) vs. 0% (11/152000) 7.6.813.1 95% (328/347) vs. 0% (328/152000) 7.6.815.1

Comment on attachment 702323 [details] [diff] [review] Block qipcap.dll, rev. 1 Review of attachment 702323 [details] [diff] [review]: ----------------------------------------------------------------- r=me

Comment on attachment 702323 [details] [diff] [review] Block qipcap.dll, rev. 1 This patch is not risky and doesn't change strings or uuids.

Comment on attachment 702323 [details] [diff] [review] Block qipcap.dll, rev. 1 There's no reason to take this for the ESRs because the crash was caused by an interface change only in FF18.

Adding qawanted to help with verification that blocklist works as expected.

I don't expect the crash to be reproduceable on nightly because we fixed the nsIPrefBranch IID.

Should we land this patch only in Release and Beta, and not in Aurora and m-c where bug 828296 is fixed?

The DLL is very likely not going to work in m-c/aurora, so I don't think there's harm in blocking it there also, and not blocking it would create a weird situation where it's loaded into a later version but not an earlier version.

https://hg.mozilla.org/releases/mozilla-aurora/rev/1f852f68169c https://hg.mozilla.org/releases/mozilla-beta/rev/dc288063bac6 https://hg.mozilla.org/releases/mozilla-release/rev/6c5332ee34d4

I tried to find Websense Endpoint in their product list, but after registering and getting a trial account I'm still waiting for their suite to finish downloading (2gb+). It looks like it is a content filtering system, and most search results were about bypassing it through a proxy. I'll look into it tomorrow when it finishes downloading.

I installed the web security part of the test suite, but after installing there was no process loading the qipcap.dll (as show using the program ListDlls). Firefox 18.0 didn't load it, and as far as I can't tell, it wasn't even found in the installation I performed. If anyone has any ideas, please let me know.

You don't need to fix because Websense did. FF 17& 18 dumped nsIPrefBranch2 interface and changed it's ID.

It's still not fixed by the DLL blocklist: nsPrefBranch::GetIntPref(char const*, int*)|EXCEPTION_ACCESS_VIOLATION_WRITE (157 crashes) 100% (157/157) vs. 0% (168/137016) QIPCAP.dll 0% (0/157) vs. 0% (1/137016) 7.3.600.0 1% (1/157) vs. 0% (6/137016) 7.6.812.1 1% (1/157) vs. 0% (6/137016) 7.6.813.1 99% (155/157) vs. 0% (155/137016) 7.6.815.1 (In reply to netpq from comment #27) > You don't need to fix because Websense did. FF 17& 18 dumped nsIPrefBranch2 > interface and changed it's ID. Where did you get this info?

> (In reply to netpq from comment #27) > > You don't need to fix because Websense did. FF 17& 18 dumped nsIPrefBranch2 > > interface and changed it's ID. > Where did you get this info? See bug 828296 - this is one of the more common causes of crashes with third-party binary software in Firefox 18. FYI, we suspect that the Norton Confidential crashes spiking in this release is connected to that as well, see the dependencies of that bug (this here is one of them, BTW).

There are still crashes in 20.0a2 and 21.0a1 after the fix of bug 828296. See https://crash-stats.mozilla.com/report/list?version=Firefox:20.0a2&version=Firefox:21.0a1&signature=nsPrefBranch%3A%3AGetIntPref%28char+const*%2C+int*%29

In that case it would be nice to figure out how the DLL is getting loaded and past our blocklist, but I don't plan on looking at this further.

(In reply to Benjamin Smedberg [:bsmedberg] from comment #31) > In that case it would be nice to figure out how the DLL is getting loaded > and past our blocklist, but I don't plan on looking at this further. Users have contacted Websense at this point and the crash remains quite low. I agree that we shouldn't devote more time to this issue specifically.

low crash depending of the organization, for us is a serious problem with my 700 users were most of them use Firefox and we must roll out websense to all the users, it crash on all Firefox versions > 17.x , tested in the beta versions as well. Do we have to recomend stop using Firefox including dev teams that produce web portals for customers and let the customers know that Firefox is no supported ? A bit ridiculous..

There are almost 30 crashes on Firefox 19 beta 5, which only got released a week ago: https://crash-stats.mozilla.com/report/list?query_search=signature&query_type=contains&reason_type=contains&range_value=4&range_unit=weeks&hang_type=any&process_type=any&signature=nsPrefBranch%3A%3AGetIntPref%28char%20const%2A%2C%20int%2A%29 I've only seen 2 post-fix crashes on Firefox 20 though, and none on Firefox 21.

(In reply to Ioana Budnar [QA] from comment #35) > There are almost 30 crashes on Firefox 19 beta 5, which only got released a > week ago: The fix is not the DLL blocklist but the Websense Endpoint update so those who crash haven't updated it.

Just in case helps to someone, I have been doing lots of testing latelly with Firefox 18.02 and different versions of websense enpoint client, the issue with the version 18.x seems to be resolved with websense endpoint version 1138 I just got today, unfortunatelly it is a preproduction version and the latest production version is 1130 to wich Firefox 18.x does not run therefore you need to request the version 1138 from websense support teams.

Considering the number of crashes in the last 4 weeks (5 on Fx 20, 0 on Fx 21 and 22), we can consider this bug fixed and verified.

Firefox 21 beta crash id bp-92f342ba-bcfd-4ad0-9daf-c609a2130429

(In reply to Swarnava Sengupta (:Swarnava) from comment #40) > Firefox 21 beta crash id bp-92f342ba-bcfd-4ad0-9daf-c609a2130429 You are using a Websense Endpoint version that doesn't contain the fix (see the whiteboard).

oh yes, i just told that user to update websense!

Sorry for jumping in here, but I believe this is the correct location for this comment. At work, we are running Websense Endpoint 1139. The current version of Firefox - 21.xx - works fine, but the latest ESR version of Firefox - 17.0.6 - crashes upon startup. If I disable Websense, ESR works normally. There may not be any action required here - just mentioning that this is still affecting current ESR builds.

(In reply to Paul Pietromonaco from comment #43) > There may not be any action required here - just mentioning that this is still > affecting current ESR builds. An extension with binary can't be compatible with 17.0 ESR and the current version if the binary has been changed after 17.0. Usually, ESR users don't need extensions.

Hello all, thank you for your efforts to collaborate. I want to provide information for direct contact regarding all product security issues in all Forcepoint products and services: I am a founding member of the Forcepoint Global Product Security Incident Response Team (PSIRT). (Forcepoint is the synthesis of Raytheon Cyber Products, Websense, Trusted Computer Solutions, Oakley Networks, Visual Analytics, Stonesoft, Sidewinder, and others) If you ever need to report a product security vulnerability, contact PSIRT@forcepoint.com. PGP Key: https://www.forcepoint.com/innovation/product-security/product-security-report-issue