Some of this could happen sooner than Mozilla 2, but I'm going to put Mozilla 2 in the summary to keep people focused away from risky work on the 1.9.x CVS tree. See bug 40848 comment 23 for the conclusion of that bug. This bug should be blocked by concrete bugs about: 1. Threads without shared memory hazards, a la Google Gears JS worker threads. 2. Rendering multi-threading, perhaps similar to the Valve Steam engine (see https://www.cmpevents.com/sessions/GD/S3728i1.pdf), to take advantage of multiple cores -- only for proven wins. 3. Plugins out of process. See bug 156493. 4. Further safe-by-design, scalable ways to exploit hardware parallelism. I'm making this meta-bug mainly so everyone who thinks threads cure cancer can come here and see the exactly problems and solutions involving parallelism, and not confuse means for ends or otherwise vent their enthusiasm for threads. Hope it works better than the last time. /be

What about DOM/nonDOM event enhancements, i.e. so that things like and equivalent to function test() { for(i = 0; i < many; ++i) { setTimeout(test, 1); } } (any kind of aggressive event posting which intersects with UI) etc. This seems like one of the major reasons due to which people complain about browser being unresponsive and many propose threading as a cure (like, recently, in http://it.slashdot.org/article.pl?sid=07/06/08/1942214). Does that belong here?

Comment 1 is unclear: what do DOM or non-DOM events have to do with setTimeout? What real-world use-case is a loop to high ("many") count creating timeouts meant to represent? Slashdot is a sewer. People everywhere are prone to put cart before horse and solve non-problems or the wrong problems. Citing such plentiful nonsense does not help. Again, what belongs in this bug are real use-cases for exploiting hardware parallelism, and ideas for addressing those use-cases -- and not random graspings at straws. /be

(In reply to comment #2) > Comment 1 is unclear: what do DOM or non-DOM events have to do with setTimeout? > Well, the idea was to represent a script code which relies much on event processing - i.e. many events fired, many events handled (either DOM events or setTimeout - anything that fires events), perhaps badly written script code, this may be one type of code that causes "unresponsiveness". The second paragraph of the response make the bug focus clear though.

Please stop. Unless you have profiled Mozilla running an "unresponsive" testcase, idle speculation about runaway JS is not wanted here. Identifying and reducing "unresponsive" testcases is welcome in new bugs, one per testcase; such new bugs might be cited here in comments. /be

(Goal for "responsive": UI should react within 20ms, in any situation.)

Reference: The above "freeze me" testcase is bug 30942 (adding as dep, if I may), but it is just the most blatantly obvious case. The "goal" is far far broader - this bug.

Is it worth explicitly defining some use-cases we would like to see solved? Problem cases, like "Java loading in one tab", or "Loading lots of Slashdot pages in background tabs". Something that would give developers a target to shoot for, something concrete that will let people declare victory over at least one particular aspect of the problem.

(In reply to comment #8) > Is it worth explicitly defining some use-cases we would like to see solved? > Problem cases, like "Java loading in one tab", or "Loading lots of Slashdot > pages in background tabs". These are fair game, even if a fast uniprocessor implementation could handle them without starving the UI. We'll figure out what can benefit from hardware parallelism after profiling and analyzing, not before. > Something that would give developers a target to > shoot for, something concrete that will let people declare victory over at > least one particular aspect of the problem. Yes, that's great -- concrete beats abstract, best is enemy of good, etc. The usual Voltaire quote, how rare. :-P What really does not make sense is prescribing "threads" as a solution for undefined or ill-defined problems, especially if there are existence proofs of responsive browsers that do not use threads. /be

(In reply to comment #6) > (Goal for "responsive": UI should react within 20ms, in any situation.) > But responsive GUI has very little to do with multi threading. Similarly working user interface with "while (true);" running in a background tab can be done without any reliance on OS-provided threads. So if the purpose of the bug is to cover the issues related to the improving the browser performance on multi core CPU, then it has nothing to do with a responsive GUI. The latter must not require presence of an extra core.

(In reply to comment #10) > working user interface with "while (true);" running in a background tab can > be done without any reliance on OS-provided threads. Well, without the process setting an alarm signal for itself, or being controlled by an external process - how, exactly? Also, even if it _can_, that doesn't mean that it _is_ ... (In reply to comment #8) I partially disagree with Brendan; I believe it is important to gather subjective user experience about the situations in which they find the browser or mail client to be non-responsive. Profiling and analysis seems to be a better tool for improving performance via parallelism rather than for avoiding freezes and hangs.

(In reply to comment #11) > Well, without the process setting an alarm signal for itself, or being > controlled by an external process - how, exactly? This can be done, for example, via good old trick of dispatching pending GUI events after JS run for a while. > Profiling and analysis seems to be a > better tool for improving performance via parallelism rather than for avoiding > freezes and hangs. In principle I agree with this as real web pages does not contain "while (true);" loops as it even does not count as DoS due to long running scripts dialogs. I.e the browser should be responsive on useful pages, not on any pages. On the other hand I suspect that always-fast-on-usable-pages can not be done without bounding the time the browser can ignore the user input. But then again, what this has to do with improving performance of multi core systems?

(In reply to comment #10) > So if the purpose of the bug is to cover the issues related to the improving > the browser performance on multi core CPU, It is. > then it has nothing to do with a > responsive GUI. The latter must not require presence of an extra core. Agreed in principle, and this is an excellent point. However (this does not contradict your point): there may be brute-force ways to use another core, e.g. for the *process* in which a plugin runs, that does indeed improve reponse over what the OS could do with threads, or our pseudo-threaded main-thread+plugin code could do, on a uniprocessor. So bug 156493 blocks this bug, and it concerns not only fault isolation but also responsiveness. /be

(In reply to comment #11) > (In reply to comment #8) > I partially disagree with Brendan; I believe it is important to gather > subjective user experience about the situations in which they find the browser > or mail client to be non-responsive. Where did I say it was not important to gather subjective user experience? I only said: not in this bug. Please pay attention. > Profiling and analysis seems to be a > better tool for improving performance via parallelism rather than for avoiding > freezes and hangs. I said so, and invited links to other bugs and comments citing other bugs, profiling, and analysis, in this bug. There seems to be a problem with subjectivity around threads or (as you noted) "threading", more than around UI responsiveness. UI responsiveness problems should be identified from symptoms (however subjectively perceived) back to causes, in separate bugs, tracked by whatever metabugs make sense. Just Not This Bug. "Threading" as a prematurely prescribed cure for lack of responsiveness seems to me to be due to hubris on the part of those who think they can hack MT code, who are convinced Mozillans are either cowardly or stupid. That's obviously rubbish, but see the recent slashdot post about Firefox 3 alpha 5 for bucket-loads of it. Bugzilla has had important bugs overrun with vague but strongly felt feelings, stubborn confusions, and continual digressions in the past. Don't aid such noise in this bug, or I'll WONTFIX it and do work in a private bugsystem. /be

(In reply to comment #12) > (In reply to comment #11) > > Well, without the process setting an alarm signal for itself, or being > > controlled by an external process - how, exactly? > > This can be done, for example, via good old trick of dispatching pending GUI > events after JS run for a while. Warning: in order to preserve run-to-completion you have to both (a) keep events well-ordered (where order matters); (b) avoid dispatching events that could nest JS in the currently running script and mutate observable state such as global variables. This is non-trivial. We started a project to do something called "window groups" in the old Netscape 5 code (Mozilla classic) -- see http://lxr.mozilla.org/classic/search?string=WindowGroup and note that the project was not completed, and the design was not proven. > On the other hand I suspect that always-fast-on-usable-pages can not be done > without bounding the time the browser can ignore the user input. > > But then again, what this has to do with improving performance of multi core > systems? Nothing -- good point, again. Note new bug alias. /be

I will chime in with the cases that I notice on a regular basis: Click on a URL with "open link in new tab". The new page has Java content. The Java plugin starts loading up, but for some reason it freezes (bug in the Java applet most often). As a result, the whole browser is frozen solid, not just the current tab. Open several windows and tabs. In one of those tabs, go to YouTube and watch several movies. Sometimes, the YouTube page freezes at the end of the movie. As a result, the whole browser gets stuck. Open a story on Slashdot (or similar site) with a very large number of comments. Until that page is fully rendered, the whole browser is slow and almost unresponsive, even though the whole OS is still fairly responsive, so that's proof that there are enough CPU cycles available. This works better on a CPU that's not the newest, let's say an Athlon XP 2000, but it's fairly visible on any CPU. I am seeing at least one instance of such a "one small portion of the browser bogs down the whole application" bug perhaps once every other week or so. I am using fairly recent versions of Firefox, both from the 1.6 and the 2.0 series. I am using Linux Fedora 6 and 7 (both 32 and 64 bits - on the 64 bit machine, I am running the 32 bit Firefox with the 32 bit Flash plugin), Linux Ubuntu 7.04 and Windows XP Pro. There doesn't seem to be a preference as to which OS or architecture is more prone to this type of bugs, however Flash on Linux does seem more fragile than other plugins. It's annoying from a user's perspective. Why should I kill the whole browser if just one portion is sick? Splitting the browser in several execution units may be a way to take advantage of the OS's multitasking capabilities, which are quite advanced on modern OSes, but it's just one way to skin this particular cat. From the user's perspective, what needs to happen is that a part of the browser does not block the rest of the same application, regardless of how that is implemented. I will continue to offer the dummy user's perspective on this issue should that be useful in fixing it. :-) Just ask.

Oh yeah - forget hardware parallelism. Even with just one CPU core, the OS is much more capable to do multitasking than the browser. Proof - the case of the huge Slashdot discussion. The whole browser is slow until the entire page is rendered, while other applications on the same single-core machine are not slow. So it's just a matter of avoiding bad interactions between tabs / windows, not a matter of distributing the load across many cores. There's no point in re-inventing the wheel and do advanced multitasking inside the browser. The OS is a much better tool for that. Hence the idea to create one execution unit per docshell.

I mailed Florin about avoiding me-toos and bike-shedding. Others tempted to vent about responsiveness issues here, please re-read comment 14 last sentence. /be

I'm sorry, I suppose I was confused about why this bug was opened in the first place. Maybe some other people are as well. My understanding is that Bug 40848 was originally opened to track the fact that a "hang" in one tab appears to "hang" the entire browser. The originally submitter felt that one solution to this could be creating a separate thread for each tab, but I don't feel that was the primary concern of the bug, merely a suggestion. Since there are a number of places where the browser could benefit from increased parallelism, I thought this bug was opened to track that "bigger issue", of which the originally-reported symptom - one "hanging" tab causing all other UI elements to "hang" - was merely one aspect. Since the other bug was marked WONTFIX at the same time this bug was opened, I had presumed that this bug was meant to handle the resolution of that bug's reported symptoms, one way or another. Is this incorrect? Brendan, I don't mean to instigate you into unleashing your WONTFIX WMD . . . but it appears as though you opened this bug to more-accurately reflect the cause of the symptoms being reported in that other bug, then closed that other bug saying this one was taking the place of it, and are now annoyed that people are trying to find a place to continue the discussion of the original "Unresponsive UI" symptoms - here, there, or wherever. So what kind of forum can we use to track peoples' "Unresponsive UI" use-cases? Would you like for us to open up a number of additional bugs, one use-case per bug? Should one "Stalled execution in one tab shouldn't cause other tab to become unresponsive" bug be opened to track *all* the "Unresponsive UI" issues, and then linked to this one? There's gotta be a way to make everyone happy... Can't we all just get along? =;)

This bug is not the successor to bug 40848 -- that bug as summarized is WONTFIX. This bug is about exploiting hardware parallelism. If you know of specific symptoms that are not already tracked by bugs about lack of responsiveness given reproducible steps (e.g., load slashdot page in a background tab), file them. They should be fixed without requiring multiple cores. If multiple cores increase throughput for some of these workloads, great -- that subset of such bugs could block this metabug. If you are new to bugzilla, not a code hacker, full of hopes and dreams, etc., that's all fine. Just don't spam this bug. /be

(In reply to comment #20) > This bug is not the successor to bug 40848 -- that bug as summarized is > WONTFIX. This bug is about exploiting hardware parallelism. But you called this bug "multi-thread and/or multi-process parallelism", and made it a meta-bug. I'm confused too...

(In reply to comment #21) > (In reply to comment #20) > > This bug is not the successor to bug 40848 -- that bug as summarized is > > WONTFIX. This bug is about exploiting hardware parallelism. > > But you called this bug "multi-thread and/or multi-process parallelism", and > made it a meta-bug. I'm confused too... That's because you assume threads are for solving responsiveness problems. I think threads are for exploiting hardware parallelism, in restricted ways (and processes are better because they don't share memory). I'll clarify the summary but I think you protest too much. Not all meta-bugs with "thread" in the summary are fair game for tracking responsiveness problems. Start from symptoms, and work back to causes, then to alternative fixes. Don't wave the thread-flag prematurely. /be

(In reply to comment #7) > Reference: The above "freeze me" testcase is bug 30942 (adding as dep, if I > may), but it is just the most blatantly obvious case. The "goal" is far far > broader - this bug. I don't think that bug should block this one. It's terribly old, not concrete, and as stated it should not require more than one CPU to solve. Adding more CPUs could help only if the critical path to servicing the event loop from an iloop of some kind could be parallelized. That's not the case in general, certainly not with JS. Bug 30942 is entirely independent of multicore utilization. /be

Brendan, yes, OK. I was confused (misunderstood this bug) for exactly the reasons Mark Bulas explained. I agree that "UI responsiveness" is a goal and threads are one solution and they should not be confused. To make this distinction clear, and direct things in the right direction, I filed tracker bug 384323 (core) and bug 384325 (apps) about UI responsiveness.

> If you are new to bugzilla, not a code hacker, full of hopes and dreams, etc. BTW, Mozilla development nicely summed up! ;-)

Hi folks, some words about threads / multiple processes: As soon as third-party code is involved (ie. the buggy flash plugin), running this within the browser process is really bad. We don't know whats happening there and why it's crashing or blocking. (My pesonal experiences on Gentoo x86: flash makes the browser unusable). So the first important step is to move off plugins to their own processes. Not just some (unobvious) sub-process, but an completely separate servlet, wich I can easily kill w/o hurting the browser. Before reinventing the wheel, we can learn from Plan9 and use their concepts. Plan9 is an heavy-distributed operating system, where many, many things run in separate servers. Servers and clients talk via the 9P2000 protocol. The next step would be, moving other problematic parts, ie. profile storage, network cache, etc, to their own servers. This isn't just about parallelism, but much simpler code flow, easier debugging and stability.

Intel has a new website with tools and tips to exploit hardware parallelism. http://www.go-parallel.com/

If google's new browser, chrome, is open-source, any chance of fitting Firefox's extension abilities on something like that framework? I don't know about anyone else -- the news was talking about chrome as being competition for IE8, but it almost seemed to address, directly, the major shortcoming of the Firefox browser design -- parallelism. Each tab can run in its own memory/cpu space, in parallel -- and extensions that fail in one tab can be 'jailed' to not affect other tabs. It seems like this strikes right at the core of FF's performance problems (which have decreased greatly, FWIW, in TB3, but TB3 can still be overloaded with too many open windows, each running tiny little automating scriplets -- none a big deal in itself, but trying to multi-task 50 open web-pages all within the constraint of 1 cpu, is a poor way to use multi-core, multi-cpu systems. I regularly see FF using "25%" of my system's CPU (1/4 cores)...and yes -- some options, like "FEBE" -- that runs a profile backup once a day (in wee morning hours), will hose the UI for the length of their run -- but they are doing real work, real file-i/o, and compressing it all -- of course it will use a full CPU, but the UI and other extensions *SHOULD* all be able to run on other cores. I'm told the big hold up is that the UI-code isn't "re-entrant"? What parts aren't reentrant? Is it possible to put the UI into it's own process or thread and work on making it non-blockable? I.e. work-callbacks would be enqueued onto a "work-queue" that one or more 'worker' threads would whittle down and and send return "done" messages? Sorry if my suggestion doesn't make sense, I'm not familiar with FF's internal limitations....

law: we're looking at multi-process now, and there have been bugs on aspects of it on file for a while. Chrome is a big boost to looking harder, but at the Firefox summit there was already a small-group meeting to discuss MP for Vista (for protected mode). And we've always wanted plugins out of process, but that work was lower priority relative to more pressing bugs. Your comment belongs in the newsgroup, not this bug. Look for a post to mozilla.dev.platform later today from jst. /be

Multi-process is also very useful for sandboxing (bug 478976), and it so happens that browser plugins are also popular attack vectors.

E10S is going well, we have threads already, this is not a useful meta-bug. The researchier project is Rust/Servo, I'll blog about that shortly. /be

I can't believe this bug is still such a problem YEARS after introduction of multicore processors... Not to mention, multi-cpu machines being available for home since ~1999, at least, if not long before (that's when I got one)... Now 12 years later...FF still executing all javascript "threads" on 1 core. Idea: Implement your own thread library based on linux source -- it automatically puts threads into separate processes, so all your threading will automatically become multi-processor! Ok, so maybe not trivial, but still, am I understanding this correctly? That things are 'multi-threaded', but due to window's design threads don't default to running on multiple CPU's? Or is there something else prohibiting this parallelization? I.e. why aren't multiple threads automatically scheduled on multiple CPU's?

> FF still executing all javascript "threads" on 1 core. Did you test that? There are threading libraries, but *using* them is hard. Problem is shared data (most of the data), and preventing concurrent access to it. This is very difficult, to the point where it makes no sense anymore. roc (Robert O'Callahan) has blogged extensively about it, please read this thoughts first. Out of process plugins and similar takes care of nasty components starving the CPU with silly loops. One process per tab takes care of nasty websites, and of many pages loading in parallel in background tabs. Coincidentally, that also separates websites and UI (although there are other solutions for that). However, the user can still wait for a single page in the foreground to render. The browser still sometimes eats 100% CPU.

(In reply to comment #33) > I.e. why aren't multiple threads automatically scheduled on multiple CPU's? OS-level threads *are* in fact automatically scheduled to run on multiple CPU cores, if they are available. To go even further, on platforms like Linux, there is very little distinction between a thread and a process -- both are understood by the kernel in an abstraction called a "lightweight process". Given that the kernel understands threads and processes to be very similar -- namely, as items-to-be-scheduled-to-run-on-one-of-my-CPU-cores -- the primary difference between a thread and a process is the sharing of operating-system-level resources. The primary example is that, when you fork a new process (instead of creating a new thread), the virtual address space is cloned (not shared) into the new process. Mozilla does not execute all JavaScript threads on one core. To clarify the terminology, a single CPU can have many cores, and the operating system can generally schedule a thread to run on any available core. In the cases where we have JavaScript executing in its own thread, those threads can simultaneously occupy as many cores as your kernel cares to give it. Given that, feel free to ask for any further clarifications!

Ben Bucksch (:BenB) 2011-07-27 12:30:08 PDT > FF still executing all javascript "threads" on 1 core. Did you test that? --- I observed it. Go into process explorer. Look at the properties. You can see multiple threads executing -- but they are all on 1 cpu. ---- Nope -- not on Windows, and that's the problem, which is why I made the 'you gotta be kidding' comment creating one's one threading lib based on the linux model. Completely NON-trivial, and said mostly tongue-in-cheek, but I would have *guessed* that linux FF didn't have the problem that exists on Windows, as windows threads don't seem to schedule themselves on multiple cpu's. With some applications, they do, I don't know the difference -- maybe some flag in the binary? If that's all it takes -- problem solved, but that would be a real big 'miss', if that's all it took to fix this prob, and it wasn't ever done - but (who cares) if it fixes it that'd be the best news since sliced bread! ---- Chris Leary [:cdleary] 2011-07-27 12:37:49 PDT (In reply to comment #33) > I.e. why aren't multiple threads automatically scheduled on multiple CPU's? OS-level threads *are* in fact automatically scheduled to run on multiple CPU cores, if they are available. --- You should stick with your knowledge base. I know linux threads were implemented on top of processes -- windows' weren't. That's why multithread benchmarks showed better perf on windows, while multi-proc apps showed better perf on linux (by VERY small amounts), but in linux, threads are about as expensive to create as procs (if not the same), while in windows, threads are cheaper (on same cpu), to create in microsecs than on linux, but process create times were greater on windows by a larger margin than the benefit on threads. On windows, threads don't automatically schedule on multi cpus (yes I know the diffs between multi-cpu, muti-core, and hyperthreads which can look like more cores. Its just that on linux -- for the longest time, it was all 'SMP', it wasn't until the multi-core procs had been out for a while and that linux got Core-specific code added in order for cache-hit optimization (among other things, but I think that's the primary reason). Very simple -- you can look at firefox running in process explorer (I wish linux had such a nice gui tool...) and see all the threads -- there will be multiple threads all running, but they all run on 1 cpu (at a time).... It's like they are all 'grouped together'.... No matter how many pages and what the javascript load on each of those pages, I never see mozilla using more than 100% of 1 core (or 16.67 of total on a 6-core)...but I would emphasize that such a claim is on windows (7, x64)...

> It's like they are all 'grouped together'.... Yes. Linux might in fact be smart and group threads of the same app together, to make better use of the L1/L2 cache (RAM bandwidth is one of the limiting factors of CPUs). Were the threads you observed all busy, i.e. used 100% of that core, for more than 1-2 seconds? If they are idle, of course they should be on the same core.

Actually, I retract that question. This is a meta-bug, not for discussion. Please file a specific bug and mark it to block this one.

(In reply to comment #36) > You should stick with your knowledge base. That's disrespectful, and certainly not in the Mozilla spirit. Even if I misunderstood what you were originally positing, a courteous clarification is all that's required. I'll be removing myself from the discussion at this point.

Meta-bug ruined by useless, rude, no-account commenting. See you do-something folks on github.com. /be

Chris Leary [:cdleary] 2011-07-27 15:35:58 PDT (In reply to comment #36) > You should stick with your knowledge base. That's disrespectful, and certainly not in the Mozilla spirit. Even if I misunderstood what you were originally positing, a courteous clarification is all that's required. I'll be removing myself from the discussion at this point. ----- I'm VERY SORRY, I didn't mean the comment to be taken with any 'sensitivity'... Unfortunately, I don't seem to get the same respect other people expect from me, so, I've lately tended to be a bit more 'snide' in some responses. It is very much not meant personally, as I don't know you. As for filing a specific bug, I had filed one, and was told to dup it to this one. I did so, and started adding comments to this one. As for 'all threads being busy'...NO, NOT all threads, but 2-3 were all competing for the 1 cpu. I HAVE killed off busy threads to see if it would 'unjam' anything, but the other threads simply took up the slack. -- so they were all competing for 100% cpu, but limited to 1 proc --- ALL the while, -- and with the main point, being that the GUI was locked up and no windows would move. That's the real point -- I wouldn't care *as much* if the various javascript tasks were compete w/each other on 1 cpu, IF the gui and other browser windows or even other TABS were responsive, but that's what brought this whole thing to my attention in the first place. And again, I apologize that my remarks were ...well, rude, I've been under a bit too much self-inflicted stress...that started with a suse 11.2->11.4 upgrade that was less smooth than any previous -- that machine hosts all data files and roaming profiles for my windows logins (yes a home setup!), but with that upgrade samba went sideways, and have profiles of old, incompat machines overwriting newer, ...etc....so a, cause b, cause c, cause d....etc....in each case, I find various bugs, and I file many, but here in the mozilla "community", I had bugs filed from 2002-2009, all responded to about a month ago -- I didn't catch the responses because they went into my 'old-bugs folder. Three weeks later, almost all of them were closed with comments about 'ms walsh not responding within 3-4weeks ' (after they didn't respond for 2-9 years?!?).... So please, understand and I hope you will make some allowances for my overall attitude being someone 'irreverent', 'flippant', and unfortunately (!not my intent), 'rude'.... it wasn't meant to be an insult, but I threw it out because it didn't seem like it was applicable to windows -- which maybe this bug was never intended to be -- but if you see the comment just prior to mine, you will see the bug that was dupped into this was about windows, and I was told to dup it here. So I assumed this was about the case on windows, since linux doesn't usually have these types of problems... (yeah, I have both...as you might gather from the suse comment)...