Closed Bug 1438214 Opened 7 years ago Closed 5 years ago

Browser hangs on infinite downloads of blobs

Categories

(Core :: DOM: File, defect, P2)

Product:
Core
Component:
DOM: File
Version:
58 Branch
Type:
defect

Tracking

()

Status:
RESOLVED DUPLICATE of bug 1472158

People

(Reporter: malwareinfosec, Unassigned)

References

(Blocks 1 open bug)

Details

(5 keywords)

Attachments

(2 files)

bomb_FF.html
(deleted), text/html
Details
blob download page freezes browser
(deleted), image/png
Details
Attached file bomb_FF.html (deleted) —
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.167 Safari/537.36 Steps to reproduce: Tested on Firefox 58.0.2 (64-bit) on Windows 7 64bit Load the attached PoC. Actual results: Firefox hangs with CPU at 100% and eventually memory runs out. Expected results: Firefox should prevent this behaviour and also allow the user to exit the tab or browser. This technique is already used in the wild (but originally targeted Chrome users): https://blog.malwarebytes.com/malwarebytes-news/2018/02/tech-support-scammers-find-new-way-jam-google-chrome/
Severity: normal → critical
Keywords: csectype-dos, hang, testcase
CC'd security as the report is related to virus-like activity.
Is there any update on this issue?
This does: setInterval(FF_jam, 0.5); and inside the interval'd function runs a while (true) loop. I don't understand why, but it doesn't seem to trigger the usual warnings ("a website is slowing down Firefox"), and hangs the parent process too. Perhaps the blob: stuff is forcing us to talk to the parent or something. Johann, can you forward this to the folks working on eviltraps to investigate further? Due warning: running the unmodified testcase in a clean-ish / test profile bricks your profile because we try to background thumbnail the page which then runs the testcase again etc. etc. (so either wipe the profile each time or wipe your places db to restore sanity - which of course you may not want to be doing)
Blocks: eviltraps
Status: UNCONFIRMED → NEW
Component: Untriaged → JavaScript Engine
Ever confirmed: true
Flags: needinfo?(jhofmann)
Product: Firefox → Core
I'm not sure what happened to our eviltraps burndown meetings, but I don't think there's much more I can do than keep this blocking the meta-bug and CC a couple of other people who are tracking eviltraps right now. I agree that this one is quite critical.
Flags: needinfo?(jhofmann)
Do you think we should add this to the ultra high priorities we have diagnosed to be worked on next?
Good grief, if this bricks your profile, it seems pretty critical. I'll investigate.
Assignee: nobody → jorendorff
Priority: -- → P1
Priority: P1 → P2
Jason, do you have any updates on this bug and your investigation (Comment 6)?
Flags: needinfo?(jorendorff)
Flags: needinfo?(sdetar)
I took a look at this bug. There are several issues at play here: * Our handling of blob URLs is very prone to spam the parent process, specifically with this message: <https://searchfox.org/mozilla-central/rev/403038737ba75af3842ba6b43b6e2fb47eb06609/dom/file/uri/BlobURLProtocolHandler.cpp#154>. The test case sends an unbounded number of these messages to the parent process, which causes the parent process to become too slow at doing critical work, specifically displaying the slow script door hanger notification. I experimented with a hacky "fix" of just commenting out that message (which is obviously not the right way to fix this bug) and with that even with a debug build, the slow script notification is displayed and can be interacted with very easily. Therefore I'm moving this to DOM: File. The right fix for this bug, I believe, is to coalesce these messages when we end up creating too many blob URLs in a very short amount of time instead of spamming the parent process like we do currently. * Per comment 3, our handling of background thumbnail capturing is prone to being tricked with infinitely running scripts. We need to unconditionally terminate long running scripts when capturing background thumbnails. I filed bug 1473344 and made it block eviltraps.
Assignee: jorendorff → nobody
Component: JavaScript Engine → DOM: File
Flags: needinfo?(sdetar)
Flags: needinfo?(jorendorff)
Just in case baku doesn't see this, I'll needinfo him.
Flags: needinfo?(amarchesini)
Bug 1472158 removes the broadcasting of BlobURLs. BlobURLs are still sent to the parent process when created and when revoked on a child process, but they are not sent to any other process. I want to see if bug 1472158 fixes this issue.
Depends on: 1472158
Flags: needinfo?(amarchesini)
I confirm. Bug 1472158 allows the parent process to show the slow-script message.
Thanks, ehsan. I owe you one.
Bug 1472158 is not going to be fixed soon: there are too many dependency to fix and we need to change dramatically the principal subsuming checks. Probably we should fine a solution for this bug in the meantime.
(In reply to Andrea Marchesini [:baku] from comment #14) > Bug 1472158 is not going to be fixed soon: there are too many dependency to > fix and we need to change dramatically the principal subsuming checks. > Probably we should fine a solution for this bug in the meantime. Are you the right person to look for such a solution? This bricks the parent process and, on a clean profile, can brick the profile. I know it's "only" rated as a sec-low, but we should really try to do something about this.
Flags: needinfo?(amarchesini)
Super duct-tape proposal, could we just throttle/rate-limit blob URL creation per document to a certain number and throw a security error otherwise? Is there any real world reason why a document needs to create 1000 blob urls (and if yes, can we put a timer on it, e.g. max 100 blob urls in 5s)?
Coming up with good heuristics in short order may be hard and bad heuristics can break real apps. How about first trying with something simple, like moving https://searchfox.org/mozilla-central/rev/403038737ba75af3842ba6b43b6e2fb47eb06609/dom/file/uri/BlobURLProtocolHandler.cpp#154 to idle dispatch to see if that relieves the pressure on the sending side a bit? Or perhaps dispatch that message with a known constant delay? Something super simple that we can just live with for one cycle is much more preferrable IMO.
See Also: → https://bugzilla.mozilla.org/show_bug.cgi?id=1306334
Summary: Browser hangs on visiting a page → Browser hangs on infinite downloads of blobs
(In reply to :Gijs (he/him) from comment #16) > This bricks the parent process and, on a clean profile, can brick the > profile. I know it's "only" rated as a sec-low, but we should really > try to do something about this. If this can brick the profile it's a bit more damaging than a typical sec-low denial of service. How "clean" does it have to be?
Keywords: dataloss, sec-low
Flags: needinfo?(gijskruitbosch+bugs)
(In reply to Daniel Veditz [:dveditz] from comment #19) > (In reply to :Gijs (he/him) from comment #16) > > This bricks the parent process and, on a clean profile, can brick the > > profile. I know it's "only" rated as a sec-low, but we should really > > try to do something about this. > > If this can brick the profile it's a bit more damaging than a typical > sec-low denial of service. How "clean" does it have to be? I mostly meant that if we try to get a background thumbnail of a page like this, that ends up trying to load the page in the background even when you've managed to close the offending page. That's... not good, obviously. We have other bugs on file about that aspect of things, too.
Flags: needinfo?(gijskruitbosch+bugs)
> How about first trying with something simple, like moving > https://searchfox.org/mozilla-central/rev/ > 403038737ba75af3842ba6b43b6e2fb47eb06609/dom/file/uri/BlobURLProtocolHandler. > cpp#154 to idle dispatch to see if that relieves the pressure on the sending ... No, this would not work. Here an example: pageA creates a blobURL and sends it via BroadcastChannel. pageB, different process, same origin, receives the blobURL (as string) via BroadcastChannel, and tries to use it. This scenario will not work if we use an idle dispatch.
Flags: needinfo?(amarchesini)
(In reply to Andrea Marchesini [:baku] from comment #21) > > How about first trying with something simple, like moving > > https://searchfox.org/mozilla-central/rev/ > > 403038737ba75af3842ba6b43b6e2fb47eb06609/dom/file/uri/BlobURLProtocolHandler. > > cpp#154 to idle dispatch to see if that relieves the pressure on the sending > ... > > No, this would not work. Here an example: pageA creates a blobURL and sends > it via BroadcastChannel. pageB, different process, same origin, receives the > blobURL (as string) via BroadcastChannel, and tries to use it. This scenario > will not work if we use an idle dispatch. What other options do we have to mitigate this issue in the meantime?
Flags: needinfo?(amarchesini)

To be honest, I don't have any temporary solution to suggest. The real fix would require an important refactoring of blob URL loading (involving nsIPrincipals comparison too). I suggest to wait, because, if this issue doesn't become extremely annoying, Fission will fix it eventually. No needs to broadcast blobURLs when we will have one process per origin.

Flags: needinfo?(amarchesini)

There's more evidence that this is used in the wild in bug 1575952, it would be great to get this reprioritized, outside of Fission.

Can we build some throttling into our child-side IPC sending side that limits the number of (same-msgid) messages we send over a given time period? And/or can we look at what other browsers have done to combat this?

Flags: needinfo?(amarchesini)

Bugbug thinks this bug is a regression, but please revert this change in case of error.

Keywords: regression
Attached image blob download page freezes browser (deleted) —

Seeing this on http[:]//macsecurities71.mac-us209[.]live
screenshot taken during browser hang/freeze attached.
It also causes general system instability

68.0.2
macOS 10.14.6

status-firefox68: --- → affected

No point tracking this on a per-release basis until there's a plan for how to fix this.

status-firefox68: affected → ---

Just encountered this bug in the latest Nightly 72. The browser became unresponsive and crashed apparently with OOM. The about:crashes page doesn't show anything.

This should be now fixed because we don't broadcast blobURLs anymore.

Status: NEW → RESOLVED
Closed: 5 years ago
Flags: needinfo?(amarchesini)
Resolution: --- → DUPLICATE

I can still reproduce this bug (or something similar) by going on the Blob URL Dos on https://eviltrap.site/ on FF75.0.1a (2020-02-16) on Linux.

Flags: needinfo?(amarchesini)

I manage to reproduce the crash but this is not related to blobURL broadcasting. It's just an OOM crash. The test tries to allocate an unlimited number of blobs and at some point, "something" crashes. For instance, I have a crash in nsURILoader::OpenURI:

#0 0x000055fd5daff5c0 in mozalloc_abort(char const*) (msg=0x7ffd5ae379f0 "out of memory: 0x", '0' <repeats 12 times>, "1000 bytes requested") at /home/baku/Sources/m/blob/src/memory/mozalloc/mozalloc_abort.cpp:33
#1 0x000055fd5daff7c5 in mozalloc_handle_oom(unsigned long) (size=0) at /home/baku/Sources/m/blob/src/memory/mozalloc/mozalloc_oom.cpp:51
...
#15 0x00007f3d54752a4d in mozilla::net::PNeckoChild::SendPDocumentChannelConstructor(mozilla::net::PDocumentChannelChild*, mozilla::dom::PBrowserChild*, IPC::SerializedLoadContext const&, mozilla::net::DocumentChannelCreationArgs const&) (this=0x55fd5ef968a0, actor=0x55fd5f985e00, browser=0x55fd5f254488, loadContext=..., args=...) at PNeckoChild.cpp:1504
#16 0x00007f3d54006558 in mozilla::net::DocumentChannelChild::AsyncOpen(nsIStreamListener*) (this=0x55fd5f985d40, aListener=0x55fe3150f680) at /home/baku/Sources/m/blob/src/netwerk/ipc/DocumentChannelChild.cpp:142

Similar crashes can be obtained using any in-memory storage API.

Flags: needinfo?(amarchesini)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: