Spawned from https://github.com/swaywm/sway/issues/4486

I have Firefox (Wayland) running on Sway 1.1.1, and I was trying to play https://marsfortherich.com

Whenever I enter the game, if I click, it "captures" the pointer and then it goes insane rotating all over the place. I believe this to be a bug in Sway.

I remember there was some protocol required for this to work, but I couldn't find an open issue on wlroots and decided to open this one to make sure I understood what was happening.

I can reproduce this issue in GNOME as well.

:yoasif, if you think that's a regression, then could you try to find a regression range in using for example mozregression?

I need these wayland features in Firefox

GrabPointer is currently disabled on Wayland with the following comment:

// Don't to the grab on Wayland as it causes a regression
// from Bug 1377084.

(the good news is that there's no need to implement the protocols directly in Firefox, GTK handles them.)

Hm, actually, this might be something else. GTK code doesn't seem to use these protocols.

Oh, so on all platforms pointer lock currently works by warping the pointer to the center all the time. This behavior is baked directly into EventStateManager unfortunately..

What's missing from this issue ?

Martin, this doesn't look like a "popup" issue.

WIP:

• probably would be ideal to have a relative movement event type that would bypass all normal handling
  and only directly go to the UIEvent thing with the position?? Not sure how to implement that, all the places
  that handle event types..
• currently this just aligns the event with the center of the window,
  which would be easy except the center in the DOM handler thing is the child's center,
  in non-fullscreen mode it's offset by the chrome height. I can't find a way to specifically get
  the chrome height in the parent nsWidget :(
• also, need to unlock the pointer when the requesting child process crashes!

Thanks a lot for looking into this, Greg! I'll shortly check back if it's indeed not possible with GDK on Wayland and then try to have a look into this.

Ok you're right, some future version of GTK4 will maybe have support for this. So we indeed need to implement this ourselves here :)

Greg: this looks very good to me already! How does the testing go, apart from the center issue?

(In reply to Robert Mader [:rmader] from comment #11)

Greg: this looks very good to me already! How does the testing go, apart from the center issue?

Well, it works for me, idk what else to say. I'd like to hear answers to all three points above. (First one would bypass the center one..)

From Matrix chat with emilio:

if so, you probably want the center of the <browser> that requests the locking, not the center of the content area in particular
you should have that in BrowserParent::RecvLockNativePointer for example
there's GetChildProcessOffset there for example, and a variety of conversion methods from parent to child coordinates

Another idea that I haven't verified yet: could we maybe reuse the point coordinates from nsWindow::SynthesizeNativeMouseEvent()? They may return coordinates relative to the window on Wayland.

(In reply to Robert Mader [:rmader] from comment #13)

From Matrix chat with emilio:

if so, you probably want the center of the <browser> that requests the locking, not the center of the content area in particular
you should have that in BrowserParent::RecvLockNativePointer for example
there's GetChildProcessOffset there for example, and a variety of conversion methods from parent to child coordinates

Thanks, turns out this is easy :)

So the patch is basically working now, the only remaining problem is that the pointer remains locked if the content process crashes (have to "alt-tab" to unlock).

Looking at the code, the only unlocking for that case currently is PointerLockManager::ReleaseLockedRemoteTarget(this) in BrowserParent::Deactivated() which just unsets a pointer. I guess that's enough for the "force mouse move" thing, but not enough for us, we need to run an actual unlock. Not sure how exactly to do this.

Can't you just call RecvUnlockNativePointer() from Deactivated()? (Maybe move the actual guts of that to a common function with a better name, but you get the idea)

Actually, yes, that works fine.

(And doing that has revealed that I forgot to nullptr the protocol object fields in nsWindow's constructor :D)

https://hg.mozilla.org/mozilla-central/rev/94659f852c66

Awesome work, Thanks! We may consider to uplift it to 89 too.

I can verify it works as expected on latest nightly:
https://noclip.website/#smg/PeachCastleGardenGalaxy
https://codepen.io/timohausmann/pen/GRZBvRq
www.justbuild.lol
www.3daimtrainer.com

\o/ great, thanks a lot!

The patch landed in nightly and beta is affected.
:unrelentingtech, is this bug important enough to require an uplift?
If not please set status_beta to wontfix.

For more information, please visit auto_nag documentation.

Given the size of the patch, where we are in the beta cycle and the fact that it is an old issue, I think this should ride the 90 train, thanks.