In EventStateManager::DispatchCrossProcessEvent(), when about to dispatch a cross-process event, if the event has an attached LayersId (in the field added in bug 1524226), use that LayersId to obtain a TabParent via TabParent::GetTabParentFromLayersId() and dispatch to that TabParent (as opposed to a TabParent obtained from nsSubDocumentFrame).

If the event carries a transform matrix in the field added in bug 1524226, apply the transform to the event coordinates before dispatching the event (or should the recipient apply the transform since the matrix is remotable anyway?).

(In reply to Henri Sivonen (:hsivonen) from comment #0)

If the event carries a transform matrix in the field added in bug 1524226, apply the transform to the event coordinates before dispatching the event (or should the recipient apply the transform since the matrix is remotable anyway?).

It appears that in e10s, the transform is applied by the parent:
https://searchfox.org/mozilla-central/source/dom/ipc/TabParent.cpp#1051

Currently, the transform is characterized by a point:
https://searchfox.org/mozilla-central/source/dom/ipc/TabParent.cpp#1980

The main problem is that the coordinates on the event are of type LayoutDeviceIntPoint. Yet, with out-of-process iframes, a content process isn't guaranteed to be merely translated but could have an arbitrary transform matrix applied to it.

I think this leaves 3 options:

1. Convert ints to floats, transform, convert back to ints. Works for normal cases where iframes are merely translated. Loss of coordinate precision when iframes are scaled and rotated.

2. Make all processes receive events in screen coordinates. Make processes aware of their top-level transform relative to the screen. Rely on whatever mechanism layout presently has for dealing with rotated and scaled in-process iframes.

3. Change events to have float-typed coordinates. Looks very invasive.

I'm tempted to go with option #1, but first I'd like to know what kind of coordinate handling out-of-process iframes currently have. rhunt, any advice?

Also, does WebRender composite chrome and the top level of content the same way as out-of-process iframes? I.e. is the top-level child process offset already baked into WebRender's matrices or is the offset between chrome and top-level content a special case outside WebRender?

Option #1 is the closest to the plan, but when writing the plan, I mistakenly thought that event coordinates were already floats.

Furthermore, it seems that only mRefPoint of the event is translated to the content process coordinate space in e10s and mLastRefPoint is never translated. It looks like mLastRefPoint is only used for DOM movementX/movementY values, which appear to be in screen space scaled to CSS pixels. Superficially, it looks like movementX/movementY are computed in the content process, yet only one input to the computation (mRefPoint) is translated. smaug, what mechanism makes the movementX/movementY correct when mRefPoint is translated to the content process coordinate space but mLastRefPoint is not?

https://hg.mozilla.org/mozilla-central/rev/58e9c32d5e53 suggests that our code is already known-wrong when there's a transform that's not strictly a translation.

Also, when the transform is not a mere translate, the "LayoutDevice" type part of LayoutDeviceIntPoint becomes a lie. :-(

(In reply to Henri Sivonen (:hsivonen) from comment #2)

Option #1 is the closest to the plan, but when writing the plan, I mistakenly thought that event coordinates were already floats.

Note that currently when we do a hit-test on the main thread across a transform, we should be maintaining precision to the nearest app unit - see code here. So if instead start rounding to nearest LayoutDevice pixel that will probably be a regression of sorts. Not sure if it will be user-detectable but I'm guessing yes.

(In reply to Henri Sivonen (:hsivonen) from comment #3)

Furthermore, it seems that only mRefPoint of the event is translated to the content process coordinate space in e10s and mLastRefPoint is never translated. It looks like mLastRefPoint is only used for DOM movementX/movementY values, which appear to be in screen space scaled to CSS pixels. Superficially, it looks like movementX/movementY are computed in the content process, yet only one input to the computation (mRefPoint) is translated. smaug, what mechanism makes the movementX/movementY correct when mRefPoint is translated to the content process coordinate space but mLastRefPoint is not?

It looks like the mLastRefPoint is updated in EventStateManager from the mRefPoint value. Presumably this happens in both the UI process and the content process. When it happens in the content process, the mRefPoint value being used is the translated one, so mLastRefPoint will also be in the translated coordinate space.

Yeah this code is pretty broken with oop-iframe's.

I just did some light reading through the code, not an expert on all the requirements. I agree that (1) seems like a good option for standing things up, but I'd also be concerned about precision.

I've got to go away for a bit, so I'll keep the ni? and try and think more about it.

(In reply to Kartikaya Gupta (email:kats@mozilla.com) from comment #6)

(In reply to Henri Sivonen (:hsivonen) from comment #2)

Option #1 is the closest to the plan, but when writing the plan, I mistakenly thought that event coordinates were already floats.

Note that currently when we do a hit-test on the main thread across a transform, we should be maintaining precision to the nearest app unit - see code here. So if instead start rounding to nearest LayoutDevice pixel that will probably be a regression of sorts. Not sure if it will be user-detectable but I'm guessing yes.

Even though the type currently says LayoutDevicePixel, the rounding would be to integer in the coordinate space of the out-of-process iframe. I assume that if the iframe has a scale transform applied to it, the integer rounding wouldn't actually correspond to rounding to device pixel precision.

But I really don't know properly how the top-level coordinate space of an out-of-process iframe works at present.

(In reply to Henri Sivonen (:hsivonen) from comment #3)

Furthermore, it seems that only mRefPoint of the event is translated to the content process coordinate space in e10s and mLastRefPoint is never translated. It looks like mLastRefPoint is only used for DOM movementX/movementY values, which appear to be in screen space scaled to CSS pixels. Superficially, it looks like movementX/movementY are computed in the content process, yet only one input to the computation (mRefPoint) is translated. smaug, what mechanism makes the movementX/movementY correct when mRefPoint is translated to the content process coordinate space but mLastRefPoint is not?

It looks like the mLastRefPoint is updated in EventStateManager from the mRefPoint value. Presumably this happens in both the UI process and the content process. When it happens in the content process, the mRefPoint value being used is the translated one, so mLastRefPoint will also be in the translated coordinate space.

Thanks. Clearing ni for smaug.

(In reply to Ryan Hunt [:rhunt] from comment #7)

Yeah this code is pretty broken with oop-iframe's.

I just did some light reading through the code, not an expert on all the requirements. I agree that (1) seems like a good option for standing things up, but I'd also be concerned about precision.

I've got to go away for a bit, so I'll keep the ni? and try and think more about it.

Thanks. I'll proceed with option 1.

Depends on D20308

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c003801590ca228ad59057e2c82f49f213c99603

https://hg.mozilla.org/mozilla-central/rev/e5162aed5ec0