Closed Bug 1222662 Opened 9 years ago Closed 9 years ago

[e10s] Mouse cursor disappears after moving it outside the Flash video area (JW Player)

Categories

(Core :: General, defect)

Unspecified
Windows 7
defect
Not set
normal

Tracking

()

RESOLVED FIXED
mozilla45
Tracking Status
e10s m8+ ---
firefox44 --- affected
firefox45 --- affected
firefox46 --- fixed

People

(Reporter: alice0775, Assigned: jimm)

References

(Depends on 1 open bug)

Details

(Keywords: regression, testcase)

Attachments

(5 files, 9 obsolete files)

(deleted), patch
jimm
: review+
kats
: checkin+
Details | Diff | Splinter Review
(deleted), text/html
Details
(deleted), text/plain
Details
(deleted), patch
bugzilla
: review+
Details | Diff | Splinter Review
(deleted), text/html
Details
+++ This bug was initially created as a clone of Bug #1222630 +++

The problem does not happen without e10s.

STR:
1) Open the testcase attachment 8684466 [details] of Bug #1222630
2) disable the mixed content protection when running the testcase
3) start the video
4) Hover the mouse over the video area
5) Move the mouse outside the video (do not move it outside content area)

Actual Result:
 mouse cursor disappears.

Expected Results:
 mouse cursor should be default(pointer)


Regression range:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=7563e84f6efc72e380aac6d43336df93ad004462&tochange=29467b3d2124

Regressed by: 	2f47017b463b	Jim Mathies — Bug 1199892 - "Mouse cursor flickers in Flash object with wmode opaque/transparent". r=roc
This WFM on Mac 10.9.5 with Nightly build from today, 20151110.
Flags: needinfo?(jmathies)
getting this out of my ni list.
Flags: needinfo?(jmathies)
Assignee: nobody → jmathies
Blocks: 1018639
Bug 1222662 - In the content process, detect when the mouse exists a plugin frame and update the content cursor accordingly.
Attachment #8689741 - Attachment is obsolete: true
Bug 1222662 - In the content process, detect when the mouse exists a plugin frame and update the content cursor accordingly; r?enn
Attachment #8689758 - Flags: review?(enndeakin)
Attached patch patch (obsolete) (deleted) — Splinter Review
Attachment #8689758 - Flags: review?(enndeakin) → review+
Attached patch patch (obsolete) (deleted) — Splinter Review
Attachment #8689758 - Attachment is obsolete: true
Attachment #8689761 - Attachment is obsolete: true
Attachment #8690128 - Flags: review+
Keywords: checkin-needed
Attached patch patch (d70778a558a7) (deleted) — Splinter Review
Attachment #8690128 - Attachment is obsolete: true
Attachment #8690129 - Flags: review+
https://hg.mozilla.org/mozilla-central/rev/d70778a558a7
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla45
Comment on attachment 8690129 [details] [diff] [review]
patch (d70778a558a7)

Approval Request Comment
[Feature/regressing bug #]:
bug 1199892
[User impact if declined]:
annoying cursor behavior with flash.
[Describe test coverage new/current, TreeHerder]:
pretty minor fix for an annoying browser bug. I'd like to get this up into aurora so it's in the next beta.
[Risks and why]: 
low risk - small number of changes in well understood code.
[String/UUID change made/needed]:
none
Attachment #8690129 - Flags: approval-mozilla-aurora?
> Last Resolved: 2015-11-24 13:59:27 PST
Is that completely OK that I still see this bug on latest Nightly 20151126030226 ?
I can confirm that the problem is still reproduced.

https://hg.mozilla.org/mozilla-central/rev/c321d84038519dcf1670d59fd2c5c00ad8a85a55
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0 ID:20151126030226
Status: RESOLVED → REOPENED
Flags: needinfo?(jmathies)
Resolution: FIXED → ---
Comment on attachment 8690129 [details] [diff] [review]
patch (d70778a558a7)

Yes it's still partially broken. When the movie isn't playing, I get proper cursor updates (switching from hand to arrow and back), but if the movie is playing the cursor hides when exiting the movie frame. Really odd.
Flags: needinfo?(jmathies)
Attachment #8690129 - Flags: approval-mozilla-aurora?
odd, so all the plumbing under the hood is working. We successfully trigger a call to ::SetCursor in nsWindow and set it to the arrow. Not sure yet how the cursor ends up getting cleared.
I hooked flash's SetCursor calls and traced them with this JWPlayer running. Flash sets the cursor async, long after they receive the last mouse move event triggered by the cursor leaving the frame. Still looking for a good fix.
Attached file testcase.html (deleted) —
Attachment #8690129 - Attachment description: patch → patch (d70778a558a7)
Attached patch patch 2 (obsolete) (deleted) — Splinter Review
Attached patch notify of set cursor calls v.1 (obsolete) (deleted) — Splinter Review
The overall problem here is that flash updates the cursor after the last mouse move via an asynchronous handler. In non-e10s, this wasn't an issue since Firefox regularly sets the cursor as you drag the mouse around content and chrome. However with e10s, we avoid repetitive sets by content because it generates a lot of ipc traffic. So flash is updating the cursor after it leaves the plugin frame, and content never sets it back due to cached value matching.

This patch tries to compensate for this by hooking SetCursor in the plugin process. When that call gets made it sets a static flag indicating the cursor changed. The instance then picks that up and sends a message over to the parent instance telling it to invalidate the cursor cache.

Unfortunately SetCursor doesn't receive an hwnd, so I had to use the static bool vs. directly communicating with the instance that set the cursor the hook. There's a risk that other instances might get a shot at servicing sFlashChangedCursor if the target instance gets wrapped up in CallWindowProc.  In practice though this didn't seem to happen, the set cursor call gets made and returns reliably to the Run() handler of the instance that set it.
Attachment #8694227 - Attachment is obsolete: true
Attachment #8694245 - Flags: review?(aklotz)
Attachment #8694245 - Flags: review?(aklotz) → review+
Attached patch windows flash follow up (obsolete) (deleted) — Splinter Review
Attachment #8694245 - Attachment is obsolete: true
Attachment #8697425 - Flags: review+
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/a3bb1bf0b330
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:46.0) Gecko/20100101 Firefox/46.0

I can still reproduce this using the test case in this bug as well as with the Hulu flash player.  Flash version 19.0.0.245.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(In reply to Trevor Rowbotham from comment #25)
> Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:46.0) Gecko/20100101
> Firefox/46.0
> 
> I can still reproduce this using the test case in this bug as well as with
> the Hulu flash player.  Flash version 19.0.0.245.

I can't reproduce in the test case when moving the mouse in and out of the video frames. Alice would you mind testing?
Flags: needinfo?(alice0775)
(In reply to Jim Mathies [:jimm] from comment #26)
> (In reply to Trevor Rowbotham from comment #25)
> > Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:46.0) Gecko/20100101
> > Firefox/46.0
> > 
> > I can still reproduce this using the test case in this bug as well as with
> > the Hulu flash player.  Flash version 19.0.0.245.
> 
> I can't reproduce in the test case when moving the mouse in and out of the
> video frames. Alice would you mind testing?

I can still reproduce the problem on Nightly46.0a1+Flash20.0.0.235.

https://hg.mozilla.org/mozilla-central/rev/06ab805e14b5e6d8a8a99c7846c8fde163a9e5d7
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0 ID:20151214122242
Flags: needinfo?(alice0775)
(In reply to Alice0775 White from comment #28)
> this seems win32 build specific problem. because i cannot reproduce on x64
> build.

I can reproduce on x64.
(In reply to Trevor Rowbotham from comment #29)
> (In reply to Alice0775 White from comment #28)
> > this seems win32 build specific problem. because i cannot reproduce on x64
> > build.
> 
> I can reproduce on x64.

yep, I can reproduce on x64 build.
(In reply to Jim Mathies [:jimm] from comment #26)
> (In reply to Trevor Rowbotham from comment #25)
> > Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:46.0) Gecko/20100101
> > Firefox/46.0
> > 
> > I can still reproduce this using the test case in this bug as well as with
> > the Hulu flash player.  Flash version 19.0.0.245.
> 
> I can't reproduce in the test case when moving the mouse in and out of the
> video frames. Alice would you mind testing?

if you cannot reproduce,  please try adding the following steps to STR comments#0.
6) enter Video's fullscreen mode and then back to normal and then moving mouse in/out.
Flags: needinfo?(jmathies)
Can either of you reproduce in 32-bit builds?
Flags: needinfo?(jmathies)
(In reply to Alice0775 White from comment #31)
> (In reply to Jim Mathies [:jimm] from comment #26)
> > (In reply to Trevor Rowbotham from comment #25)
> > > Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:46.0) Gecko/20100101
> > > Firefox/46.0
> > > 
> > > I can still reproduce this using the test case in this bug as well as with
> > > the Hulu flash player.  Flash version 19.0.0.245.
> > 
> > I can't reproduce in the test case when moving the mouse in and out of the
> > video frames. Alice would you mind testing?
> 
> if you cannot reproduce,  please try adding the following steps to STR
> comments#0.
> 6) enter Video's fullscreen mode and then back to normal and then moving
> mouse in/out.

This is a different bug from what was described in comment 0. If this is the new test case, we'll close this bug out and file a new one on full screen issues.
I'll file a follow up on 64-bit. If there's an issue with fullscreen please file a fresh bug.
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Blocks: 1232633
Blocks: 1232636
Blocks: 1232640
(In reply to Jim Mathies [:jimm] from comment #32)
> Can either of you reproduce in 32-bit builds?

I can still reproduce this bug on both 32bit/64bit builds with STR in comment#0
Filed Bug 1222662
No longer blocks: 1232640
(In reply to Jim Mathies [:jimm] from comment #33)
> (In reply to Alice0775 White from comment #31)
> > (In reply to Jim Mathies [:jimm] from comment #26)
> > > (In reply to Trevor Rowbotham from comment #25)
> > > > Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:46.0) Gecko/20100101
> > > > Firefox/46.0
> > > > 
> > > > I can still reproduce this using the test case in this bug as well as with
> > > > the Hulu flash player.  Flash version 19.0.0.245.
> > > 
> > > I can't reproduce in the test case when moving the mouse in and out of the
> > > video frames. Alice would you mind testing?
> > 
> > if you cannot reproduce,  please try adding the following steps to STR
> > comments#0.
> > 6) enter Video's fullscreen mode and then back to normal and then moving
> > mouse in/out.
> 
> This is a different bug from what was described in comment 0. If this is the
> new test case, we'll close this bug out and file a new one on full screen
> issues.

Filed Bug 1232640
No longer blocks: 1232636
(In reply to Jim Mathies [:jimm] from comment #37)
> *** Bug 1232636 has been marked as a duplicate of this bug. ***

Is this still reproducible in 32-bit builds for you alice?
Flags: needinfo?(alice0775)
Ok, osrry, missed comment 35.
Status: RESOLVED → REOPENED
Flags: needinfo?(alice0775)
Resolution: FIXED → ---
Tracy can you try to reproduce this. It's on windows, 32-bit firefox, moving the mouse cursor in and out of the video frame over content. The cursor apparently disappears. I currently can't reproduce on Win7.
Flags: needinfo?(twalker)
I can also reproduce this in 32-bit builds, so, this is not isolated to 64-bit.  Jim, in response to your question in Bug 1232633 comment 2, the videos work for me in 64-bit nightly, you just need to disable mixed content blocking for them to show up.

If you still can't get the JW Player test case to work, I can reproduce this with the flash player on Hulu.com.  The problem with mousing out of the video on Hulu only seems to occur on 3 sides of the video, though; top/right/bottom.  There doesn't seem to be an issue with mousing out of the left side, when using Hulu, which is odd.
(In reply to Trevor Rowbotham from comment #41)
> I can also reproduce this in 32-bit builds, so, this is not isolated to
> 64-bit.  Jim, in response to your question in Bug 1232633 comment 2, the
> videos work for me in 64-bit nightly, you just need to disable mixed content
> blocking for them to show up.
> 
> If you still can't get the JW Player test case to work, I can reproduce this
> with the flash player on Hulu.com.  The problem with mousing out of the
> video on Hulu only seems to occur on 3 sides of the video, though;
> top/right/bottom.  There doesn't seem to be an issue with mousing out of the
> left side, when using Hulu, which is odd.

Trevor, would you mind posting your about:support?
Attached file support.txt (deleted) —
As requested in comment 42.
Ok, so I have traced the lineage of this bug.  Hopefully, this helps you zero in on a fix.

First regression (Sorry, mozregression failed to bisect inbound for this one):

Last good revision: 688f821edcd4 (2014-11-12)
First bad revision: ab137ddd3746 (2014-11-13)
Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=688f821edcd4&tochange=ab137ddd3746

Regressed by: Bug 669200

I filed bug 1141718 on this issue.

The first regression was then subsequently fixed by Bug 1018639. See:

First good revision: ffd1b16f058b7ddb85f7f003a993fb730d7d2016
Last bad revision: c24fb4429029af2e88cd7db3a144802a113ccf0b
Pushlog: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=c24fb4429029af2e88cd7db3a144802a113ccf0b&tochange=ffd1b16f058b7ddb85f7f003a993fb730d7d2016

The second time this regressed (which is what this bug is referring to):

See Alice's regression range in comment 0.
Flags: needinfo?(jmathies)
I can reproduce on Win7. The video has to be playing for the bug to reproduce.  Prior to the video playing, mouse behavior works as expected.
Flags: needinfo?(twalker)
Flags: needinfo?(jmathies)
Attached patch native cursor cache (obsolete) (deleted) — Splinter Review
alternative approach.  try builds -

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3714ad5895d4
Attached patch native cursor cache (obsolete) (deleted) — Splinter Review
Attachment #8700731 - Attachment is obsolete: true
Blocks: 1231944
Using the build from the try run, this significantly improves the situation for me when it comes to the mouse cursor disappearing completely, however, there are still a few bugs with this.

I observed that the cursor will sometimes get stuck as the cursor last set by flash after having moused out of the video in the testcase.  This seems to occur most frequently when the video is paused and when the mouse exits from the right side of the video.

I have also observed, that with some effort, I can still get the cursor to disappear altogether when mousing out of the flash video.  This occurred most frequently with the Hulu flash player as opposed to the testcase in this bug.
Attached patch disable cursor setting cache (deleted) — Splinter Review
This fix disables caching on Windows this reverting things back to single process behavior.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=04f43a85e747
The patch from comment 49 appears to resolve the major issues for me.

The only minor glitch that I observed with this build is what I had mentioned in Bug 1141718 comment 1, in which there appears to be some sort of dead zone at the very edges of the plugin where the mouse cursor can still disappear.  In the testcase in this bug, this dead zone only seems to be a problem while the video is playing, but the cursor will correct itself within 1-2 seconds even if left in the dead zone.  In the case of the Hulu flash player, the cursor will stay hidden as long as it is left within the dead zone regardless of whether the video is playing or not, but it will use the correct cursor as soon as it is moved out of the dead zone.

In summary, the patch in comment 49 resolves the issue in comment 0 and the minor glitch I mentioned above is likely a separate issue altogether (perhaps an issue with plugin boundaries that is creating some sort of dead zone between the plugin and content?).
Attached file testcase w/borders (deleted) —
I think the dead area is within the video frame, potentially over the last outer pixel.
Comment on attachment 8700780 [details] [diff] [review]
disable cursor setting cache

It's unfortunate we have to land this because it will generate a lot of async ipc traffic when you move the mouse, but I see no other way around this bug. This landing will include the backout of the SetCursor hooks I landed previously.
Attachment #8700780 - Flags: review?(aklotz)
Attachment #8700780 - Flags: review?(aklotz) → review+
Attachment #8700732 - Attachment is obsolete: true
Attachment #8697425 - Attachment is obsolete: true
note to tree managers, this checkin should include a backout of the previous landing (cset a3bb1bf0b330) and the new patch in 'disable cursor setting cache'. thanks!
Keywords: checkin-needed
Blocks: 1233612
No longer blocks: 1233612
Comment on attachment 8690129 [details] [diff] [review]
patch (d70778a558a7)

This one already landed.
Attachment #8690129 - Flags: checkin+
(In reply to Jim Mathies [:jimm] from comment #54)
> note to tree managers, this checkin should include a backout of the previous
> landing (cset a3bb1bf0b330) and the new patch in 'disable cursor setting
> cache'. thanks!

Done, but please verify I did it right.
https://hg.mozilla.org/mozilla-central/rev/3f4b6a6542ec
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Depends on: 1259930
Depends on: 1299069
No longer depends on: 1299069
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: