Closed Bug 1070775 Opened 10 years ago Closed 9 years ago

25% perf regression Scroll slow and laggy due to landing Bug 1002992

Categories

(Core :: Graphics: ImageLib, defect)

32 Branch
x86_64
Windows 7
defect
Not set
normal

Tracking

()

RESOLVED WORKSFORME
Tracking Status
firefox32 --- affected
firefox33 --- affected
firefox34 --- affected
firefox35 --- affected
firefox-esr31 --- unaffected

People

(Reporter: alice0775, Assigned: tnikkel)

References

()

Details

(Keywords: perf, regression)

+++ This bug was initially created as a clone of Bug #1070772 +++ This bug is for #2 regression below. Steps To Reproduce: 1. Open testcase index.html 2. Run the following code in ScratchPad with chrome privilege (function() { function scroll(event) { var top = content.document.documentElement.scrollTop || content.document.body.scrollTop; if (top < 6000) { var evt = content.document.createEvent("KeyboardEvent"); evt.initKeyEvent ("keypress", true, true, window, null, null, null, null, 40, 0) content.document.dispatchEvent(evt) } else { content.removeEventListener("scroll", scroll, false); content.scrollTo(0,0); alert(new Date().getTime() - start); } } content.focus(); content.scrollTo(0,0); var start = new Date().getTime(); content.addEventListener("scroll", scroll, false); scroll(); })(); Actual Results: Scroll slow and laggy There are atleast 2 regressions. #1: 200% slower than previous, Regressed by Bug 994081 #2: 25% slower than the previous, Regressed by Bug 1002992 #1 Regression window(m-i) Good: 6000 msec L:\trunk\2014\06\inbound\firefox inbound 06-Jun-2014 1811 1402051313 https://hg.mozilla.org/integration/mozilla-inbound/rev/4511d9ac1000 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0 ID:20140606104153 Bad: 13000 msec https://hg.mozilla.org/integration/mozilla-inbound/rev/79624417d247 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0 ID:20140606104333 Pushlog: http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=4511d9ac1000&tochange=79624417d247 Regressed by: Bug 994081 - Convert imgFrame to store and act on a Moz2D SourceSurface instead of a Thebes gfxASurface #2 Regression window(m-i) Bad: 13000 msec https://hg.mozilla.org/integration/mozilla-inbound/rev/e2714481c419 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0 ID:20140606191036 Bad: 16000 msec https://hg.mozilla.org/integration/mozilla-inbound/rev/47fcf180a0b9 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0 ID:20140606192436 Pushlog: http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=e2714481c419&tochange=47fcf180a0b9 Regressed by: Bug 1002992 - use a bare frame tree walker for image visibility Regarding #1 regression, See Bug 1070772.
Correct: 2. Run the following code in ScratchPad with chrome privilege (function() { function scroll(event) { var top = content.document.documentElement.scrollTop || content.document.body.scrollTop; if (top < 11000) { var evt = content.document.createEvent("KeyboardEvent"); evt.initKeyEvent ("keypress", true, true, window, null, null, null, null, 40, 0) content.document.dispatchEvent(evt) } else { content.removeEventListener("scroll", scroll, false); content.scrollTo(0,0); alert(new Date().getTime() - start); } } content.focus(); content.scrollTo(0,0); var start = new Date().getTime(); content.addEventListener("scroll", scroll, false); scroll(); })();
Summary: Scroll slow and laggy → 25% perf regression Scroll slow and laggy
Timothy, is this regression expected?
Assignee: nobody → tnikkel
No, bug 1002992 should have improved perf and produced the same results. It's possible that the new code is slower in some cases (usually it's 3+ times faster) at it's job, or maybe we are doing the image locking/unlocking slightly differently and it leads to worse behaviour (more decoding) in this case. I'll investigate.
(In reply to Timothy Nikkel (:tn) from comment #4) > maybe we are doing the image locking/unlocking > slightly differently and it leads to worse behaviour (more decoding) in this > case. I don't think it's this. We get the same list of visible images. So I downloaded the inbound builds (mac) for the push for bug 1002992, and the push immediately before it and then ran the code from comment 2. I got the following results. For the push immediately before bug 1002992: 4058, 3989, 4050, 3972, 4063, 3997. avg 4021.5, stddev 39.9337. For the push of bug 1002992: 3912, 3932, 3905, 3942, 3946, 3921. avg 3926.33333, stddev 16.45195. From these numbers it looks like, if anything, bug 1002992 had a positive impact on this testcase. Alice, do you think you could double check your results?
Flags: needinfo?(alice0775)
I have to restart the browser for every measurement. For the push immediately before bug 1002992 (m-i cset e2714481c419): 13288 14664 13667 14787 13821 13915 For the push of bug 1002992 (m-i cset 47fcf180a0b9): 17336 19085 17652 16949 18218 17269 Without a doubt, bug 1002992 causes regression of the performance.
Flags: needinfo?(alice0775)
I redid my measurements, this time starting the browser fresh for each measurement. But I got pretty much the same results. Without bug 1002992: 4020, 4127, 3981, 4004, 4069, 4003. avg 4034. stddev 54.29549. With bug 1002992: 4037, 3908, 3957, 3987, 3980, 3920. avg 3964.83333. stddev 47.39374. Other than me testing on mac, and you on Windows, what other differences might there be that might be causing this? I don't easy access to a Windows machine right now to test this unfortunately.
Bas, let's get measurement on Windows as well, see if we can reproduce Alice's results locally?
Flags: needinfo?(bas)
In local build last Good: 3254fa7b1c51 First Bad: ce548c7d8fa2 Regressed by: ce548c7d8fa2 Timothy Nikkel — Bug 1002992. Part 5. Use a custom frame tree walker to find visible and almost visible images instead of using the heavier weight display list infrastructure. r=mats
I don't have any better ideas for how to investigate this so I created some try builds with some changes that might have an effect. Alice, would you mind testing these? http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-ad4ea4469570/try-win32/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-83667965d4d1/try-win32/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-cf97714b9b48/try-win32/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-a673077f61e0/try-win32/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-bd4e45d9809f/try-win32/ Also, it might be interesting to see what your results look like it you don't restart between every measurement with a regular nightly. If they are the same or different that will be a useful data point. And I assume using a fresh profile doesn't change the results? Thanks!
Flags: needinfo?(alice0775)
ad4ea4469570 13501 12451 13186 12976 12971 13152 83667965d4d1 13077 13134 12906 12721 13174 12963 cf97714b9b48 13195 13200 12691 13252 13059 13153 a673077f61e0 13287 12803 13028 12998 13019 12853 bd4e45d9809f 14363 14841 14675 14634 14313 14096
Flags: needinfo?(alice0775)
FYI, Some improved in Nightly35.0a1, but not Aurora34.0a2. Progression window(m-i) Bad about16000: https://hg.mozilla.org/integration/mozilla-inbound/rev/80164e15bd54 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140912024553 Improved about13000: https://hg.mozilla.org/integration/mozilla-inbound/rev/c232720a2847 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140912025257 Pushlog: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=80164e15bd54&tochange=c232720a2847 Improved by: Bug 1062723 I am not sure, Bug 1062723 is related to Bug 1002992 or not. And large regression Bug 1070772 still exist.
Thank you. (In reply to Alice0775 White from comment #11) > ad4ea4469570 > 13501 12451 13186 12976 12971 13152 This looks about the same as mozilla-central. This made all image decoding async. > 83667965d4d1 > 13077 13134 12906 12721 13174 12963 This looks about the same as mozilla-central. This made all image decoding async, except if there was work completed off main thread we notified about the finished work synchronously. > cf97714b9b48 > 13195 13200 12691 13252 13059 13153 This looks about the same as mozilla-central. This made all image decoding so some decoding synchronously. > a673077f61e0 > 13287 12803 13028 12998 13019 12853 This was plain mozilla-central, so we had a baseline. > bd4e45d9809f > 14363 14841 14675 14634 14313 14096 This is slower than mozilla-central. This made us use the old display list based image visibility pass. So this shows that the frame walker based image visibility pass is faster. So this contradicts your earlier results. I remember that mozregression used to (still does?) get confused by the distinction between pgo and not pgo, maybe that's what happened here?
(In reply to Alice0775 White from comment #12) > https://hg.mozilla.org/integration/mozilla-inbound/ > pushloghtml?fromchange=80164e15bd54&tochange=c232720a2847 > Improved by: Bug 1062723 > > I am not sure, Bug 1062723 is related to Bug 1002992 or not. > And large regression Bug 1070772 still exist. I think this speed up is unrelated to bug 1002992. But good to know!
(In reply to Timothy Nikkel (:tn) from comment #10) > Also, it might be interesting to see what your results look like it you > don't restart between every measurement with a regular nightly. If they are > the same or different that will be a useful data point. https://hg.mozilla.org/mozilla-central/rev/1735ff2bb23e Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0 ID:20140925030203 12842 5201 5101 5150 5131 5034 https://hg.mozilla.org/releases/mozilla-aurora/rev/e970bc96f8b5 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0 ID:20140925004001 20210 21434 20884 21584 Nightly35.0a1: It become fast if I do not restart browser. Aurora34.0a2: painfully slow :( regardless of restarting browser > And I assume using a fresh profile doesn't change the results? Thanks! Almost same.
(In reply to Timothy Nikkel (:tn) from comment #13) > I remember that mozregression used to (still does?) get confused by the > distinction between pgo and not pgo, maybe that's what happened here? I did not use mozregression tool. I used zip file of tinderbox build[1] and its archive[2]. [1] http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32/ [2] http://inbound-archive.pub.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32/
(In reply to Timothy Nikkel (:tn) from comment #13) > Thank you. > > (In reply to Alice0775 White from comment #11) > > a673077f61e0 > > 13287 12803 13028 12998 13019 12853 > > This was plain mozilla-central, so we had a baseline. > > > bd4e45d9809f > > 14363 14841 14675 14634 14313 14096 > > This is slower than mozilla-central. > This made us use the old display list based image visibility pass. > > So this shows that the frame walker based image visibility pass is faster. > So this contradicts your earlier results. I do not think so. Because, Bug 1062723 is incuded in mozilla-central now, so I think the pref regression of Bug 1002992 is hidden by Bug 1062723.
Okay, to get rid of the confounding effect of bug 1062723 I made two more pushes to try to test. Both pushes are based on http://hg.mozilla.org/mozilla-central/rev/59d4326311e0 which is just a little before bug 1062723. One has the frame walker, one uses the display list. They should be ready in about an hour from when I post this comment. https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-0a3bfbbfd895/ https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-b0591923b6a3/
Flags: needinfo?(alice0775)
0a3bfbbfd895 14429 14663 15413 13896 13904 14446 b0591923b6a3 15841 13608 14996 15498 15100 15119
Flags: needinfo?(alice0775)
(In reply to Alice0775 White from comment #19) > 0a3bfbbfd895 > 14429 14663 15413 13896 13904 14446 This is display list builder. avg 14458.5, stddev 561.74612. > b0591923b6a3 > 15841 13608 14996 15498 15100 15119 This is frame walker. avg 15027, stddev 763.24989. The standard deviation to is too big to conclude that either one is faster than the either, so it looks like a tie. Unless we get more data.
I don't have any better ideas for how to track down what is going on here, so I produced some try builds to try to bisect when bug 1002992 had a regressing effect on mozilla-central by disabling it on different revisions. These should be ready in an hour. https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-a120b09a9856/ https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-4d914299ad38/
Flags: needinfo?(alice0775)
(In reply to Timothy Nikkel (:tn) from comment #22) > Sorry, build error. Use these links instead. > > https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com- > a120b09a9856/ > https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com- > 171ce5d00304/ a120b09a9856 15926 17181 15769 16355 15709 16150 171ce5d00304 14014 15147 14180 14891 13863 14177
Flags: needinfo?(alice0775)
8526961d91b4 20642 19962 20028 20205 20283 20722 3e0c3f227c02 17516 16198 17338 16686 16203 17798 8e3e717be941 13746 13541 13864 13347 13032 13697 b808965b53ab 15237 16192 14942 15955 16658 15258
Flags: needinfo?(alice0775)
4945d624571e 5047 5166 6030 4783 5130 5546 b36d78faa22a 5629 5429 6392 5979 5663 5513 67f71fcc749c 17675 16775 17028 17132 17257 16397 c7ebd51d36fe 13846 13663 14129 13696 13647 13005 0f43f94793e6 14330 14320 14627 15063 13813 15030 1e3de144bed7 16549 16868 17157 17196 17563 16277 2307e701a7df 19267 19002 19629 17937 18357 18810 4760961f1ac7 16677 16391 16331 16114 17556 16724
Flags: needinfo?(alice0775)
13d4527a56cd 13063 13762 13791 14080 14430 13686 1acb4e806af6 15520 15229 14525 14792 15379 15626 387c0103854c 16738 16706 16409 16957 17313 16456 bcbacf1949c2 16846 15251 15357 15011 16158 16666 caa32647a226 9935 10806 10435 10157 10034 9824 ec46f3a855f8 15165 14304 13240 13946 13630 14743 e0222b14df4c 14080 14029 13537 13478 14213 15457 29ae16b82a27 10768 11449 11384 10300 12363 11473 e6e1735ec21e 13547 13014 14030 13813 14594 14313 99777486f0f9 23031 22968 22431 25169 24515 23288 430e8b1364e3 15796 15819 15413 15588 14653 15057
Flags: needinfo?(alice0775)
Thanks again. A bunch more builds. Some are there already, the rest will be there in an hour. http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-ca39e671836d/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-1c7170a407cf/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-a89aa46e0a93/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-201e72654bec/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-8587be73c923/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-df104ebaa26b/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-fd3c566228d6/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-3136b8867e64/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-de2c479c75e1/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-7d2735b73de9/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-5de96d3dffd5/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-0b858bc58791/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-edfab9de74cd/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-9ab2c0d41928/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-f6e6b9191d0e/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-c5463b044ccb/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-b33c443b079c/ http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tnikkel@gmail.com-3cfe1d898bdc/
Flags: needinfo?(alice0775)
ca39e671836d 14880 15561 14967 15578 14695 15734 1c7170a407cf 13367 13433 12993 14233 13558 14493 a89aa46e0a93 17213 19150 17479 18748 18641 17716 201e72654bec 16000 18952 17799 18675 17332 16622 8587be73c923 17049 19203 17576 19185 18345 17623 df104ebaa26b 18115 18114 17999 17668 18675 17211 fd3c566228d6 17509 15463 17783 17344 16861 17270 3136b8867e64 16774 17633 16926 16951 18074 16278 de2c479c75e1 17748 18624 18617 17061 18776 18977 7d2735b73de9 20061 20013 19943 19700 18990 20960 5de96d3dffd5 19453 19867 19395 18983 20498 19233 0b858bc58791 16055 14614 15356 14952 15549 18758 edfab9de74cd 14238 14631 14080 14993 13370 16974 9ab2c0d41928 17734 20233 18227 17731 17502 17757 f6e6b9191d0e 19422 18959 17586 19501 19077 17632 c5463b044ccb 16733 16067 17041 16743 15462 16323 b33c443b079c 16603 15457 16792 15277 16004 17360 3cfe1d898bdc 29193 30627 28840 30301 31468 30010
Flags: needinfo?(alice0775)
0b4cbfa5dde3 14733 14700 15304 14674 13901 14150 d89aa5d3d790 15398 16512 16845 15411 16894 16261 3b3a4b0b69b7 30489 31455 29959 29417 30385 31278 c5abab9cdddc 19991 19254 19005 19129 19125 19060 61dc5da60bf0 20006 20341 20027 19508 20060 19843 78e0bdb5c558 17530 18135 17150 19259 18396 18368
Flags: needinfo?(alice0775)
615458fdbe45 18645 18107 18829 17641 19932 18353 8fc2ce5dc00c 18339 19411 19044 18483 18559 18354 77d775ca5ac2 17621 18549 18581 18275 17885 18738 bfa27e268aa4 18877 18691 17914 19030 18790 17187 da4e7565750f 17965 18437 18135 18329 18223 19092
Flags: needinfo?(alice0775)
I've learned a few things, not as many as I would have liked though. I've learned that the display list builder and the frame walker are a tie on current mozilla-central. I've learned that before bug 994081 landed, the display list builder and the frame walker are also tie. I learned that on no platform can I reproduce what Alice is seeing (that the frame walker is a regression when it landed); I have tested on mac, linux, and Windows. I learned that doing both the frame walker and the display list builder fixes the regression (and it doesn't matter which order you do them in). I learned that you don't even need to do the whole display list builder pass, it is enough to build a display list, and then iterate over it (and not do anything while you iterate over it). However if you don't iterate over the list it seems to be a huge regression, worse then the original regression. I learned a lot of things have no effect.
7c5fb9366e61 19422 20523 19046 20348 20234 19573 5ebd432a91df 18851 19319 19062 18451 19306 19484 e5b541c88d39 18115 17668 18114 18407 16575 17065 d45f61fbfeee 4222 4503 4539 4404 4486 4258 34d8effab9a8 4500 4983 4717 4947 4567 4649
Flags: needinfo?(alice0775)
Alice, do you think you could provide a profile of this running on your machine? Instructions are here https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Profiling_with_the_Built-in_Profiler
Flags: needinfo?(alice0775)
(In reply to Timothy Nikkel (:tn) from comment #39) > Alice, do you think you could provide a profile of this running on your > machine? Instructions are here > https://developer.mozilla.org/en-US/docs/Mozilla/Performance/ > Profiling_with_the_Built-in_Profiler > Ctrl+Shift+2 - Take a profile and launch Cleopatra to view it Nothing happens...
Flags: needinfo?(alice0775)
(In reply to Alice0775 White from comment #40) > (In reply to Timothy Nikkel (:tn) from comment #39) > > Alice, do you think you could provide a profile of this running on your > > machine? Instructions are here > > https://developer.mozilla.org/en-US/docs/Mozilla/Performance/ > > Profiling_with_the_Built-in_Profiler > > > > Ctrl+Shift+2 - Take a profile and launch Cleopatra to view it > > Nothing happens... It seems like nothing happens, but if you just wait a little while a new tab will eventually open with the profile. (It happens to me too, and it was super confusing until I figured that out.)
Flags: needinfo?(alice0775)
And make sure the Firefox window has focus (not the scratch pad window), and make sure to close the modal alert dialog. If that doesn't work just click on the profiler button in the tool bar to bring up the profiler panel with buttons for start/stop/analyze.
cfe0518ab437 19138 19400 19297 18664 18715 18397 ecf236d04bbe 17633 18004 18832 17802 17434 17301 24124b11d610 17250 17551 16784 17900 17351 18600 37b216c563db 18498 19547 18420 19197 19180 20051
Flags: needinfo?(alice0775)
Alice, I'm curious to know if you see any regression when the pref image.high_quality_downscaling.enabled is set to false. ie did bug 1002992 have any effect? Same for bug 994081. And if current nightly is the same or different from the start of June.
Flags: needinfo?(alice0775)
(In reply to Timothy Nikkel (:tn) from comment #46) > Alice, I'm curious to know if you see any regression when the pref > image.high_quality_downscaling.enabled is set to false. ie did bug 1002992 > have any effect? Same for bug 994081. And if current nightly is the same or > different from the start of June. Nightly(2014-06-06) 5725 5554 5648 6193 5757 5991 Nightly(2014-06-06) with image.high_quality_downscaling.enabled = false 5183 5413 5522 4540 5398 5211 Nightly(2014-06-07) 15622 15827 16165 16318 14925 16527 Nightly(2014-06-07) with image.high_quality_downscaling.enabled = false 4169 4335 4321 4322 4322 4219 Nightly(2014-10-01) 11751 13463 12412 11471 11673 12406 Nightly(2014-10-01) with image.high_quality_downscaling.enabled = false 14157 14134 14085 14104 14200 15350
Flags: needinfo?(alice0775)
Summary: 25% perf regression Scroll slow and laggy → 25% perf regression Scroll slow and laggy due to landing Bug 1002992
FYI, I field a bug. Bug 1076725 - Scrolling speed has become 200% slower after landing Bug 1044702 under condition of image.high_quality_downscaling.enabled = false
This looks under control.
Flags: needinfo?(bas)
Alice, do you think you could re-test this testcase to see if the situation has improved for you? There have been a lot of image related changes over the last year.
Flags: needinfo?(alice0775)
I can no longer reproduced on Latest Nightly45 without e10s
Status: NEW → RESOLVED
Closed: 9 years ago
Flags: needinfo?(alice0775)
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.