Sometime between 5PM on June 8 and 5PM on June 10, the performance in several areas has degraded significantly. Here are two examples: 1. From the menu go to tasks->privacy->forms-manager->interview Page loads in three (3) seconds with my June 8 build Page loads in twenty-three (23) seconds with my June 10 build 2. Fill in any field on this form Echo of characters typed is instantaneous with my June 8 build Echo comes significantly after character is typed with my June 10 build I have no idea where the problem lies but since scc checked in massive string changes at about that time, I'll start by assigning this to him. If I'm wrong, then please reassing to whomever you think would be responsible for this.

I did some poor-man's profiling (by pausing the debugger repeatedly and looking at the stack trace). It seems like were spending *tons* of time in the table's frame construction code. I'll attach a typical stack trace. All I can say is that there sure are a lot of fields on this form: I'm skeptical that this is scc, and think it's more likely to be recent editor changes or something.

I think morse was referring to form prefill and capture, which certainly is very slow -- in fact, prefill goes into an infinite loop now (how slow can you get?). I have fixes for some of this, but the code really is sub-optimal, and very hard to maintain, for reasons too great to go into here.

I should add that the slow loading of INTERVIEW.HTML should get its own perf bug.

Maybe this bug isn't about wallet? Sorry, my comments are out of line. Adjust summary, add URL.

No, I wasn't referring to form prefill and capture. I was referring to form loading as well as text entry into fields. Should I make a separate bug for the text entry part? Yes, this form does have a lot of fields. But that's good -- it makes an excellent test vehicle for performance. It loads up nearly instantly in 4.x (about a second), took about 3 seconds in seamonkey as of June 8, and now takes about 23 seconds.

Slow text entry has been broken out into a separate bug. That bug number is 42471.

Putting on [nsbeta2-] radar for beta2. Performance work is for nsbeta3...putting on that keyword.

Please reconsider the nsbeta2- rating on this. Performance improvements are one thing but this is a regression that just occurred and its on the order of 7 times slower. That's pretty serious! Also it makes the wallet interview form practically useless.

I put a profile of the page loading at: http://www.inttek.com/~jlnance/mozilla/hiprof/14Jun00/42451.html.gz Let me know if I can run another one for you.

PDT would like more info. If this is related to scc's string work, it should go away in tomorrow's build. sairuh, can you retest?

As mentioned in another bug, no string changes _whatsoever_ went in between June 8th and 5pm on June 10th. The closest string change was at 5:49pm on the 10th. It seems unlikely this is a string change.

nsbeta2+, sounds like it needs a new owner. cc'ing mjudge

Re-assigning to mjudge as per general agreement.

setting to m17, setting to P1 - critical

ok so the bug is assigned to me. If I find that it is in layout do I reassign it to them?

I will attach the snipit of functions calls sorted by function+descendants percentage of total time. This is from a quantify run that I can also provide.

Even better would be too make two quantify runs. One from before the regression, and one from after the performance degradation. The comparison should at least tell what's going on during those 23 seconds.

added fix by date in status

See my comment in 42471 about confusion re date in status line.

Changing status date format to eliminate confusion

8/2--one day before bits on the wire--is a ridiculous date. If that's the date then you should lobby to bump this one out of PR2 or we have to slip, this gives no time for QA or for any last minute problems.

*** Bug 44589 has been marked as a duplicate of this bug. ***

Marc, please take a look as we discussed. I'm not sure why anonymous content is being used in the Ender light widget, but a lot of time is being spent in CreateAnonymousFrames. This bug is also related to 42471.

I have created a new testcase that is a form with hundreds of text input fields and it is very slow to load, about 25 seconds on my machine in Viewer. When the form has radio controls instead of text controls it only takes about 3 seconds to load (you control which table gets loaded by a style rule in the HEAD - setting the .txt selector to display:none prevents the frames for the div containing the text controls from being created, setting that rule on the radio div prevents the radio button containing div from being created). The problem is clearly with the text controls, and that makes me think that this 'regression' may have been caused by the enabling of the EnderLite control for text inputs. Thought the original URL has a large table, tables have nothing to do with this problem. Also, waterson's comment of 2000-06-13 21:22 impliend that there may be an issue with the table frame construction code, but that seems to be a symptom of the problem and not a cause since we get the pathetic behavior without tables. I'll now get into what is so slow about creating a input-text control vs. an input-radio control... My guess is that the EnderLite control is much much slower to create, but I do not know why. If anybody can give me some information on how the EnderLite control is created, or how to turn off EnderLite for input-text controls that would help a lot (CC'ing mjudge for EnderLite help). Also, clearing ETA until I can get a more realistice handle on the problem and it's scope.

I hope I'm mistaken, but it looks like each input=text has the following frames: nsGfxControlFrame2@038B7B48 next=038BCAB8 {0,23010,2460,300} [state=20c04024] sc=038C7CC8< GfxScroll(div)(-1)@038B7C64 {30,30,2400,240} [state=20c04004] sc=038BA5F8< ScrollPortFrame(div)(-1)@038B7D08 [view=03EE1240] next=038B7DAC {0,0,2400,240} [state=20c06004] sc=03A51678< Area(div)(-1)@038B7C14 [view=03EE2D60] {0,0,2400,240} [state=000d6000] sc=03A519A0(i=1,b=0)< line 038BCA90: count=1 state=inline,clean,NOT Impacted,[0x500] {0,0,1,240} < Frame(br)(0)@038BCA64 {0,0,1,240} [state=00004024] > > > ScrollbarFrame(scrollbar)(-1)@038B7DAC [view=03EDC730] next=038BC494 {0,240,2400,0} [state=20c06024] sc=038BFA88< Box@038B7E44 next=038BC0F0 {0,0,0,0} [state=20c04024] sc=03A52070< ImageBox@038BC04C {0,0,0,0} [state=00004024] > SliderFrame(slider)(-1)@038BC0F0 [view=03EDDDC0] next=038BC35C {0,0,0,0} [state=20c06024] sc=038BBC90< Box@038BC1B0 {0,0,0,0} [state=20c04024] sc=03A526C0< Spring@038BC240 {0,0,0,0} [state=00004024] ImageBox@038BC27C {0,0,0,0} [state=00004024] Spring@038BC320 {0,0,0,0} [state=00004024] > > Box@038BC35C {0,0,0,0} [state=20c04024] sc=038BD6D0< ImageBox@038BC3F0 {0,0,0,0} [state=00004024] > > ScrollbarFrame(scrollbar)(-1)@038BC494 [view=03EDF380] {2400,0,0,240} [state=20806024] sc=038BFA88< Box@038BC52C next=038BC664 {0,0,0,0} [state=20c04024] sc=038BDD20< ImageBox@038BC5C0 {0,0,0,0} [state=00004024] > SliderFrame(slider)(-1)@038BC664 [view=03EE0BB0] next=038BC8D0 {0,0,0,0} [state=20806024] sc=038BBC90< Box@038BC724 {0,0,0,0} [state=20804024] sc=038BE3F0< Spring@038BC7B4 {0,0,0,0} [state=00004024] ImageBox@038BC7F0 {0,0,0,0} [state=00004024] Spring@038BC894 {0,0,0,0} [state=00004024] > > Box@038BC8D0 {0,0,0,0} [state=20c04024] sc=038BEA40< ImageBox@038BC964 {0,0,0,0} [state=00004024] > > > > For the testcase, the following is the frame size dump: Frame Type Count TotalSize MinSize MaxSize AvgSize ---------- ----- --------- ------- ------- ------- AreaFrame 185 14060 76 76 76 BRFrame 183 8052 44 44 44 BlockFrame 3 228 76 76 76 CanvasFrame 1 56 56 56 56 LineBox:block,big 1 60 60 60 60 LineBox:block,small 2 80 40 40 40 LineBox:inline,smal 258 10320 40 40 40 ScrollFrame 370 20720 56 56 56 TextFrame 222 13464 60 64 60 TextInputFrame 184 10304 56 56 56 ViewportFrame 1 60 60 60 60 *** Total *** 5110 262404 which suggests that we have 2 scrollframes per control? And 1 Area, 1 BR, 1 linebox, 1 text, and 1 textinput = that's a lot of frames.

I'm not sure how flexible the ender lite widget is, but there should be no scroll bars for <input type=text>. Only <textarea> needs them. <input type=text> is supposed to be a small amount of text fitting inside a limited space. I have seen scroll bars pop up, and they destroy the layout. It would be better (if possible) to do the simple horizontal scrolling with arrow keys, as the native widgets do.

I did a bit of analysis in Quantify this afternoon. Turns out we're creating a native widget for each <input> element. Talked with evaughan a bit about this: 1. NeedsClipWidget() has its logic backwards. He'll check that fix in. 2. NeedsClipWidget() is never being called. evaughan sez that kmcclusk should have set up nsScrollBoxFrame to support operation without a native widget. I don't understand all the details. Bottom line: ender lite was not supposed to create native widgets; it is; this is accounting for ~30-40% of the time. Who should fix this? (For those interested in the details, reflowing the native widgets accounts for 25% of the time: this is dominated by time spent moving the native windows around. Another 10% or so is taken up just to create and initialize the native widget for each text field.) pink: You should be aware that we initialize drag & drop on every single text field. (Maybe we do that because this is a native widget?) This is accounting for another 10% of the time, because we monkey around in OLE.

Zoinks! Kevin is on vacation for another week and a half - I'll see if Rod S. can help out with the View portion in the meantime. It will be interesting to see how slow the page feels with a 30-40% speedup - something tells me there is more to fix than the native widget since the pages is about 10X too slow.

I looked into this through LXR and it looks like evaughan made some massive changes to nsScrollPortFrame.cpp on 6/22, basically undoing the changes that Kevin made to get rid of the native widget for ender light controls. Since evaughan's change on 6/22 looks like a complete re-write of nsScrollPortFrame I have no clue how to fix it myself, and it clearly looks like the bug is a regression due to the changes made on 6/22, under the comment of 'Autoscrolling menus feature landing'. Sending this to evaughan to take a deeper look at the changes - specifically, what happened to the logic where we used to call IsInsideFormControlFrame to determine if we need a widget? That method was removed and now we always create the widget. Here are the bulk of the changes that Kevin made to get native widgets out of EnderLight: http://bonsai.mozilla.org/cvsview2.cgi?diff_mode=context&whitespace_mode=show&fi le=nsScrollPortFrame.cpp&root=/cvsroot&subdir=mozilla/layout/html/base/src&comma nd=DIFF_FRAMESET&rev1=3.33&rev2=3.34 Here are the changes evaughan made that regressed this: http://bonsai.mozilla.org/cvsview2.cgi?diff_mode=context&whitespace_mode=show&fi le=nsScrollPortFrame.cpp&root=/cvsroot&subdir=mozilla/layout/html/base/src&comma nd=DIFF_FRAMESET&rev1=3.34&rev2=3.35 Once the widget is out again, we may still have additional performance problems. If so, send it back to me and I'll look into possible issues with layout's interaction (ie. reflow madness).

ccing kin

scrollportframe was not rewritten it was just moved and renamed to scrollboxframe because it is a box frame. Scrollportframe simply became a simple subclass. Looks like something broke in the process. I'll take a look.

I did some high tech stopwatch profiling: Eric's patch (attached above) speeds up stuff by 2x. (Took about 4-5s instead of 10-11s on my machine to layout 250 continguous text fields.)

Can you get some timing on what the patch does to the URL sited at the head of this report. That url used to load in three seconds on June 8 and fell to 23 seconds on June 10.

I just applied the patch and did some timing. The INTERVIEW.HTML page cited above now loads in 15 seconds instead of 23. But prior to this regression it used to load in 3 seconds. Furthermore, the first time I ran the browser after applying the patch, it asserted and then crashed on startup. I didn't capture the stack trace thinking I would get it on the next try. Unfortunately I have not been able to duplicate the assertion or crash on future tries.

OK, after repeating the test about five times (just entering browser and then exiting) I was able to reproduce the assertion followed by crash at startup. Stack traces shown below. I then udid the patch and repeated the test ten more times. Was not able to get either an assertion or a crash on startup. That doesn't prove conclusively that the assertion/crash is caused by the patch, but it does give food for thought. stacktrace at assertion: NTDLL! 77f76274() nsDebug::Assertion(const char * 0x01ce4914 ??_C@_0DJ@KMGL@You?5can?8t?5dereference?5a?5NULL?5nsC@, const char * 0x01ce4958 ??_C@_0N@NHHF@mRawPtr?5?$CB?$DN?50?$AA@, const char * 0x01ce4968 ??_C@_0BO@LIAM@?4?4?2?4?4?2dist?2include?2nsCOMPtr?4h?$AA@, int 649) line 246 + 13 bytes nsDebug::PreCondition(const char * 0x01ce4914 ??_C@_0DJ@KMGL@You?5can?8t?5dereference?5a?5NULL?5nsC@, const char * 0x01ce4958 ??_C@_0N@NHHF@mRawPtr?5?$CB?$DN?50?$AA@, const char * 0x01ce4968 ??_C@_0BO@LIAM@?4?4?2?4?4?2dist?2include?2nsCOMPtr?4h?$AA@, int 649) line 342 + 21 bytes nsCOMPtr<nsIDocument>::operator->() line 649 + 34 bytes nsXBLEventHandler::ExecuteHandler(nsXBLEventHandler * const 0x024ec8c0, const nsString & {"keypress"}, nsIDOMEvent * 0x02539f34) line 592 + 41 bytes nsXBLEventHandler::KeyPress(nsIDOMEvent * 0x02539f34) line 143 + 40 bytes nsEventListenerManager::HandleEvent(nsIPresContext * 0x01fe4c00, nsEvent * 0x0012f730, nsIDOMEvent * * 0x0012f510, nsIDOMEventTarget * 0x024225c0, unsigned int 7, nsEventStatus * 0x0012f69c) line 1094 + 23 bytes nsXULElement::HandleDOMEvent(nsXULElement * const 0x024225b0, nsIPresContext * 0x01fe4c00, nsEvent * 0x0012f730, nsIDOMEvent * * 0x0012f510, unsigned int 1, nsEventStatus * 0x0012f69c) line 3350 PresShell::HandleEventInternal(nsEvent * 0x0012f730, nsIView * 0x02027f80, nsEventStatus * 0x0012f69c) line 3906 + 45 bytes PresShell::HandleEvent(PresShell * const 0x020276f4, nsIView * 0x02027f80, nsGUIEvent * 0x0012f730, nsEventStatus * 0x0012f69c, int & 1) line 3841 + 23 bytes nsView::HandleEvent(nsView * const 0x02027f80, nsGUIEvent * 0x0012f730, unsigned int 28, nsEventStatus * 0x0012f69c, int & 1) line 782 nsViewManager2::DispatchEvent(nsViewManager2 * const 0x020262f0, nsGUIEvent * 0x0012f730, nsEventStatus * 0x0012f69c) line 1389 HandleEvent(nsGUIEvent * 0x0012f730) line 69 nsWindow::DispatchEvent(nsWindow * const 0x02027e04, nsGUIEvent * 0x0012f730, nsEventStatus & nsEventStatus_eIgnore) line 560 + 10 bytes nsWindow::DispatchWindowEvent(nsGUIEvent * 0x0012f730) line 581 nsWindow::DispatchKeyEvent(unsigned int 131, unsigned short 0, unsigned int 13) line 2133 + 15 bytes nsWindow::OnChar(unsigned int 13, unsigned int 13, unsigned char 1) line 2257 nsWindow::ProcessMessage(unsigned int 258, unsigned int 13, long 1835009, long * 0x0012fab8) line 2685 + 51 bytes nsWindow::WindowProc(HWND__ * 0x017106c2, unsigned int 258, unsigned int 13, long 1835009) line 829 + 27 bytes US stack trace at crash nsXBLEventHandler::ExecuteHandler(nsXBLEventHandler * const 0x024ec8c0, const nsString & {"keypress"}, nsIDOMEvent * 0x02539f34) line 592 + 53 bytes nsXBLEventHandler::KeyPress(nsIDOMEvent * 0x02539f34) line 143 + 40 bytes nsEventListenerManager::HandleEvent(nsIPresContext * 0x01fe4c00, nsEvent * 0x0012f730, nsIDOMEvent * * 0x0012f510, nsIDOMEventTarget * 0x024225c0, unsigned int 7, nsEventStatus * 0x0012f69c) line 1094 + 23 bytes nsXULElement::HandleDOMEvent(nsXULElement * const 0x024225b0, nsIPresContext * 0x01fe4c00, nsEvent * 0x0012f730, nsIDOMEvent * * 0x0012f510, unsigned int 1, nsEventStatus * 0x0012f69c) line 3350 PresShell::HandleEventInternal(nsEvent * 0x0012f730, nsIView * 0x02027f80, nsEventStatus * 0x0012f69c) line 3906 + 45 bytes PresShell::HandleEvent(PresShell * const 0x020276f4, nsIView * 0x02027f80, nsGUIEvent * 0x0012f730, nsEventStatus * 0x0012f69c, int & 1) line 3841 + 23 bytes nsView::HandleEvent(nsView * const 0x02027f80, nsGUIEvent * 0x0012f730, unsigned int 28, nsEventStatus * 0x0012f69c, int & 1) line 782 nsViewManager2::DispatchEvent(nsViewManager2 * const 0x020262f0, nsGUIEvent * 0x0012f730, nsEventStatus * 0x0012f69c) line 1389 HandleEvent(nsGUIEvent * 0x0012f730) line 69 nsWindow::DispatchEvent(nsWindow * const 0x02027e04, nsGUIEvent * 0x0012f730, nsEventStatus & nsEventStatus_eIgnore) line 560 + 10 bytes nsWindow::DispatchWindowEvent(nsGUIEvent * 0x0012f730) line 581 nsWindow::DispatchKeyEvent(unsigned int 131, unsigned short 0, unsigned int 13) line 2133 + 15 bytes nsWindow::OnChar(unsigned int 13, unsigned int 13, unsigned char 1) line 2257 nsWindow::ProcessMessage(unsigned int 258, unsigned int 13, long 1835009, long * 0x0012fab8) line 2685 + 51 bytes nsWindow::WindowProc(HWND__ * 0x017106c2, unsigned int 258, unsigned int 13, long 1835009) line 829 + 27 bytes USE

My apologies, the assertion/crash has nothing to do with the patch. After many more tries I was able to get the crash to occur with the patch removed. Just opened bug 45358 on that. So ignore my comments in this report about the assertion/crash on startup.

fixed. -r waterson

No, it's better but it's not fixed. That site used to load in 3 seconds and now it takes 15 seconds. Admittedly that's better than 23 but it's still far from what it was before the regression.

Well, evaughan has fixed the problems that he's responsible for. Let's find a new owner for the bug.

Re-profiled the 250-text field case with evaughan's changes. Frame construction now accounts for 84% of the time; reflow for 6%, and the rest is spread across the parser and painting and who knows what. So, The Next Biggest Thing appears to be nsXBLService::LoadBindings() (30% of the total time; 37% of frame construction). This is split evently between nsXBLBinding::InstallEventHandlers() and nsXBLBinding::GenerateAnonymousContent(). Hyatt, wanna take a crack at this?

No.

I just talked to hyatt about this. He thinks much of the time is being spent constructing and computing style for scrollbar frames (~10 frames per scrollbar, 2 scrollbars per widget) that are just hidden. This is probably the time spent in XBL. He thinks the API that was added to hide the scrollbars should be changed so it doesn't make them at all. This could be some work since the scroll frame code (or whatever, I'm not the expert) may assume scrollbars. He also thinks further performance gains would probably come from lazy-initialization of the text widgets like the old ender-medium used to do, i.e., keeping them as a DIV until they are used.

Ok, here are my suggestions. Single line input fields are creating scroll frames. Scroll frames always create scrollbars. mjudge and I hacked the scrollframe so that the scrollbars are collapsed and never shown. This obviously sucks. This is why XBL is on the stack, although it isn't XBL that's slow. It's properly indicating that about 25 scrollbar, button, thumb, slider frames, etc. etc. per input field should be constructed. This is not all underneath the GenerateAnonymousContent stack, and so it probably accounts for much of the frame construction cost. The fix is to stop the scrollbars from ever being made at all. For single-line inputs, I wonder if we even need scroll frames in our way... perhaps we should just be using a scrollable view. Alternatively, frame construction should be patched so that a scroll frame can be made without scrollbar widgets. It seems to me that a slick fix for this problem would be to optimize the GFX scroll frame so that it doesn't make scrollbars until it has to show one of them. Right now it always makes scrollbars, even when you never show any scrollbars. If the scroll frame were smarter about creating the scrollbars lazily, then this would improve performance all across the board. I'm reassigning this bug to evaughan for consideration, since I believe the best fix can be made in the scroll frame code.

Most of the frames created on this url are related to scrolling. <input type=text> is not supposed to be scrolled, except that left/right arrows should do primitive horizontal scrolling. <textarea> needs scrollbars but <input type=text> does not and its layout gets messed up if the scroll bars appear. So Hyatt's suggestion is a good one. But I think we should go one step further for <input type=text> and not even have a scroll frame, unless ender relies on that to do the simple arrow type scrolling.

Ok the best fix for this is to use a "scrollbox" wrapped around the div in the editor instead of overflow auto. Overflow auto on a div creates a gfxscrollframe, 2 scrollbars, and a scrollbox that is wrapped around the div. If you just place a scrollbox around the div instead we can remove 3 frame constructions per ender widget. So in you anonymous content creation method. Create a "scrollbox" and place the "div" inside it. And see if things get any better.

Downgrading to beta2- because this is now just a performace optimization and we would not pull from the wire to fix it.

It's always been "just a performance optimization" and was still given the pdt+ rating. Problem is it's a major performance regression. See my comments of 6-14.

Sorry guys, but you don't get to change a plus to a minus, or vice versa, only the directors seem to have that authority outside the triage meeting. The proper process here is to clear the status whiteboard to force another triage (which I'm doing now). Please add any further comments needed to support your positions, being very clear about the effect of this defect to common users. You are also welcome to present your cases in person, weekdays in StarTrek between 1-2 PM.

Putting on [nsbeta2-] radar. Not critical to beta2. Adding "nsbeta3" keyword for consideration of a fix for that milestone.

nsbeta3+, reassign to hyatt for m18

Fix checked in. Page now loads in 2-3 seconds for me.

I hate to keep sounding like a pessimist, but it's still not fixed. On my machine it is now loading in 8 seconds. Admittedly that's an improvement and we keep getting better (since this bug report was opened we've gone from 23 to 15 and now to 8). But on June 8 I was able to load this page in 3 seconds.

Ok, well, my part of it is done. Reassigning to editor.

Are you using a debug build? I just tried optimized, and the page loaded in about 1.5 seconds. Given that your page is an extreme example (it has a huge # of textfields on it), I think this is probably good enough for beta3. Anyway, will leave it up to editor to decide what to do with it.

On the bright side, hyatt's checkin today for this bug considerably improved the situation for bug 42471. See comment that I'm about to add there.

Yes, I'm using debug build. I'll try it with an optimized build later this evening (after I pull and build an optimized tree).

I just built a commercial tree and timed it. THREE SECONDS!!!! Things sure are much faster when you don't spit out all the diagnostic information. Thanks Dave. I'm reclosing this now.

Wow. Looks great dudes...nice job! Marking VERIFIED FIXED on: - LinuxRH62 2000-09-07-08-M18 Commercial - Win98 2000-09-07-08-M18 Mozilla - MacOS86 2000-09-07-04-M18 Commercial

i just tried this and the given page can't be found. ck? how did you verify?

You can get to this page from the menu as follows: tasks->privacy->form-manager->interview

wow, that's one heck of an interview...spouse's mother's maiden name!? Geez! ;)

Try opening a joint account at an on-line brokerage house and that's a question you will definitely be asked.