@media should have support for interaction features as described at http://dev.w3.org/csswg/mediaqueries4/#mf-interaction. According to http://status.modern.ie/mediaquerieslevel4interactionmediafeaturespointerandhover these are already supported by Chrome and Opera.

Those features now have been implemented in all other browsers.

FWIW, Blink's implementation can be find here: https://chromium.googlesource.com/chromium/src.git/+/master/ui/base/touch/ in the touch_device_*.cc files.

Given how much crippled, bloated, "My first UI" design infuriates me when viewing formerly useful sites like PayPal and My eBay where I'm trying to get work done, I'll be using these to support mobile devices. As much as I love Firefox (I use it exclusively on the desktop), everyone else implements these and my need for a practical, usable desktop experience trumps brand loyalty. Until Firefox Mobile supports these, I'll be responding to any reports of touch-unfriendly Firefox Mobile rendering on my sites with RESO WONTFIX.

This is annoying from a web development perspective because it seriously limits the ability to use :hover effectively across platforms. On most mobile browsers, :hover makes no sense, and is actually the exact opposite behavior that you'd want in many cases. There are a lot of weird hacks to avoid the :hover state from being activated forever after touching that item on the screen, but they mostly involve not using hover unless the hover media query is active. Since Firefox does not respond to these, there ends up being no :hover functionality available for Firefox desktop users.

In case this does get worked on at some point, note that the CSS4 MQ spec changed recently to remove the "on-demand" value for "hover/any-hover". See related bug on Chromium/Blink https://bugs.chromium.org/p/chromium/issues/detail?id=654861 and Edge https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/9308676/ As for the comments here about the usefulness (or lack thereof) of the spec itself, see for instance https://dev.opera.com/articles/media-features/ (tl;dr: it has the potential to be misused, unless developers use it defensively)

Just FYI: Firefox supports the non-standard -moz-touch-enabled media feature. I'm using it in my app. https://developer.mozilla.org/en-US/docs/Web/CSS/Media_Queries/Using_media_queries#-moz-touch-enabled So, not yet tested but this should work: > @media (-moz-touch-enabled: 1), (pointer: coarse) { > // Rules for touch screens > }

(In reply to Kohei Yoshino [:kohei] from comment #9) > Just FYI: Firefox supports the non-standard -moz-touch-enabled media > feature. I'm using it in my app. > > https://developer.mozilla.org/en-US/docs/Web/CSS/Media_Queries/ > Using_media_queries#-moz-touch-enabled > > So, not yet tested but this should work: > > > @media (-moz-touch-enabled: 1), (pointer: coarse) { > > // Rules for touch screens > > } Thank you!

Note: Gmail's touch enabled display density requires this.

Here's a patch which implements the pointer, hover, any-pointer, and any-hover features, based on what Blink is doing for Window, Linux, Android, and OSX (save for some improvements and avoiding a bug where any-hover:none triggers on Linux when the spec says it should not). WebKit has a pretty rudimentary implementation, and neither it nor Blink have any fancy logic for determining what the "primary" device is, so I just did what they do in this code, reasoning that it will be sufficient for interop/webcompat. Note that while I've tested the patch on all the devices I have (OSX, Linux, Windows and Android), I don't have a touchscreen monitor, Android-compatible mouse, a Windows tablet/phone, or other pen/stylus type input devices to test with, so this could use some broader QA. Worse, I have no clue how I would add automated tests for this (I see no real test-coverage for similar features like MaxTouchPoints either). Kohei, Felip, you wouldn't happen to know who I should ping for info on that, would you?

Sorry I'm not sure.

Jim, do you know who might be able to help verify what should be done re: testing for this patch?

There's a new-ish set of tests called "web-platform-tests" that are meant to be interop tests that can be run on any browser. It would be nice if you can figure out how to write and use this type of test here, although I don't know how dynamic is that test suite to know if you can force the touch mode for some tests.. You should send an Intent to Implement e-mail to mozilla.dev.platform talking about your goal of implementing this feature, and there you'll probably be able to get more useful suggestions on how to approach testing for this.

Thanks Felipe, I'll definitely send an intent... but I don't see any web platform tests for these features (nor do I see how they could test these abilities with decent coverage).

No way to automate such tests in wpt currently.

(In reply to Thomas Wisniewski from comment #14) > Jim, do you know who might be able to help verify what should be done re: > testing for this patch? We have apis in nsIDOMWindowUtils [1] that simulate touch events for testing, those call down into widget and trigger actual input events that tests then use. If need be you could add new api if we need new event simulations (assuming platforms provide a way to fire them). [1] http://searchfox.org/mozilla-central/source/dom/interfaces/base/nsIDOMWindowUtils.idl#348

Note that testing here wouldn't involve simulating touch events (or any events), but simulating the actual "environment" (however the browser determines, by interfacing with the OS, that a touchscreen is present, or whether or not any mouse/stylus is present, etc)

(In reply to Patrick H. Lauke from comment #19) > Note that testing here wouldn't involve simulating touch events (or any > events), but simulating the actual "environment" (however the browser > determines, by interfacing with the OS, that a touchscreen is present, or > whether or not any mouse/stylus is present, etc) Yeah, therein lies the rub. I'm guessing the best route to go would be to go through my code (and its dependencies), and replace each OS-specific call with a call to a wrapper which decides (during runtime) whether to make that OS call or to mock it (so each test-case can mock a different device config). I'm just not sure how to implement that. My guess would be, roughly: - add a new method nsIWidget::MockSystemPointers, called by a similar new method nsIDOMWindowUtils::MockSystemPointers, which would set a static/class variable nsWidget::SystemPointerProfile denoting which configuration to mock (or to not mock at all). - have my code in this patch (and its dependencies) not directly call OS-specific hooks, but rather call wrapper methods (MockableSystemCallsGTK::gdk_display_get_default, MockableSystemCallsWindows::GetSystemMetrics, etc). - have those methods read nsWidget::SystemPointerProfile, and if a mock was set, not call the OS-specific routines, but rather just return mocked values. But I'm probably missing something there, plus I'm not sure if there are better/OS-specific ways we should mock these things instead.

Jim, do you think my approach in comment 20 would be fine?

The only relevant testing we do of -moz-touch-enabled is in test_media_queries.html[1] to ensure it is parseable. (It's included in a torbug fingerprinting case as well, but I'm not sure that's relevant here) Adding appropriate cases there seems like it could satisfy the testing requirement. [1]: http://searchfox.org/mozilla-central/source/layout/style/test/test_media_queries.html

Thanks, chutten. I've just sent an intent to implement with an explanation that the only tests I plan to add are CSS parsing ones. I'll refresh my patch with those tests as soon as I can.

Tagging this for fingerprinting. As mentioned in the email, we'd like this to check nsContentUtils::ShouldResistFingerprinting() and if that returns true, behave like you described: > Bear in mind that I'm operating on the presumption that we would want anti-fingerprinting to always return values for Windows/OSX/Linux that convey there is only a fine, hover-capable pointer (a mouse), and on Android to convey that there is only a coarse, non-hover-capable pointer (a touchscreen).

(In reply to Thomas Wisniewski from comment #20) > (In reply to Patrick H. Lauke from comment #19) > > Note that testing here wouldn't involve simulating touch events (or any > > events), but simulating the actual "environment" (however the browser > > determines, by interfacing with the OS, that a touchscreen is present, or > > whether or not any mouse/stylus is present, etc) > > > Yeah, therein lies the rub. I'm guessing the best route to go would be to go > through my code (and its dependencies), and replace each OS-specific call > with a call to a wrapper which decides (during runtime) whether to make that > OS call or to mock it (so each test-case can mock a different device > config). I'm just not sure how to implement that. > > My guess would be, roughly: > - add a new method nsIWidget::MockSystemPointers, called by a similar new > method nsIDOMWindowUtils::MockSystemPointers, which would set a static/class > variable nsWidget::SystemPointerProfile denoting which configuration to mock > (or to not mock at all). > - have my code in this patch (and its dependencies) not directly call > OS-specific hooks, but rather call wrapper methods > (MockableSystemCallsGTK::gdk_display_get_default, > MockableSystemCallsWindows::GetSystemMetrics, etc). > - have those methods read nsWidget::SystemPointerProfile, and if a mock was > set, not call the OS-specific routines, but rather just return mocked values. > > But I'm probably missing something there, plus I'm not sure if there are > better/OS-specific ways we should mock these things instead. Something along these lines I guess. What do we need to simulate? Looks like 1) touch screen css detection 2) some sort of pointer type (none, coarse, fine) Note the touch info comes from look and feel info vs. widget. You might chat with someone who know dom better, there may be a way to inject this higher up in the stack.

Hi Thomas, do you plan to finish this out? Any way I can help? Thanks!

I was hoping to finish it sometime by the end of Q2. But if you have time before that then, the patch needs tests and a special-case for resist-fingerprinting mode where it only mimics a single standard mouse pointer (as per comment 25). If you'd like to finish it sooner, please feel free! I can at least try to rebase it soon, if that would help.

Oh, that sounds awesome actually. One thing I have noticed is that we don't react to dynamic changes to it. I don't know how easy it is, I guess it's another bunch of platform dependent code, and detecting moves across screens. Or maybe it's not too bad, looking at nsDeviceContext::CheckDPIChange and friends.

Are other browsers dynamic in that way? (If they aren't, then we can potentially defer making it dynamic to a follow-up patch).

I've not had a chance to test, but (based on this bug https://bugs.chromium.org/p/chromium/issues/detail?id=442418) it should be dynamic for Chrome (at least on Android).

Yeah... In any case I agree it's probably worth doing it as a followup, if only to keep this patch under control in terms of complexity.

Mass bug change to replace various 'parity' whiteboard flags with the new canonical keywords. (See bug 1443764 comment 13.)

I did introduce a new table named nsCSSKeywordAndBoolTableEntry in bug 1365045. It would be useful media features that are evaluated in Boolean Context, I believe.

Taking. I had a brief chat with Thomas about he can take time to finish this, unfortunately he can't, so I will take over this. As for automation test cases, we can implement GetPointerCapabilities() function for headless widget and make the implementation is able to be controlled by a nsIDOMWindowUtils function. Then the test runs only for the headless mode. I think we don't need to run the tests on all platform generally (except for tests checking dynamically changes).

(In reply to Hiroyuki Ikezoe (:hiro) (PTO on Aug 20) from comment #37) > As for automation test cases, we can implement GetPointerCapabilities() > function for headless widget and make the implementation is able to be > controlled by a nsIDOMWindowUtils function. Then the test runs only for the > headless mode. After looking at attachment 8904713 [details] [diff] [review] more closely, I start thinking we don't need to add the new nsIDOMWindowUtils function. Instead we should use nsLookAndFeel and switch the corresponding pref value in the tests. Because, in attachment 8904713 [details] [diff] [review], we get system pointer information though nsIWidget, as you know on E10S we have PuppetWidget for content processes and the PuppetWidget normally delegates its functionality to the parent widget via IPC calls. But yes, in the case we get the system pointer information we don't need to delegate it because we can get the information in the child processes too, so in attachment 8904713 [details] [diff] [review] nsWindow::GetPointerCapabilities (which is called from PuppetWidget) has '#ifdef XP_WIN ...', that's odd. And unfortunately this way doesn't work on non-E10S, actually as for Android (i.e. non-E10S) the dedicated widget (nsWindow for Android) gets the system information directly in the patch.

I finally managed to find some time to help push this one closer to the finish line. Here's a fresh patch, still lacking tests, but rebased and supporting resist-fingerprinting mode as well as the boolean context. The patch appears to be working locally for me on OSX, Linux, Android, and Windows 10 builds, but I'm doing a fresh try-run to confirm whether I've missed anything obvious before I head out on PTO: https://treeherder.mozilla.org/#/jobs?repo=try&revision=5f01e058936d609cd9fbd6281e56b336f9a57564

(In reply to twisniewski from comment #39) > The patch appears to be working locally for me on OSX, Linux, Android, and > Windows 10 builds, but I'm doing a fresh try-run to confirm whether I've > missed anything obvious before I head out on PTO: > https://treeherder.mozilla.org/#/ > jobs?repo=try&revision=5f01e058936d609cd9fbd6281e56b336f9a57564 Thank you, Thomas! A confusing point here is that, as far as I can tell, 'pointer' and 'hover' should be matched for the primary pointer device, whereas 'any-pointer' and 'any-hover' should be matched for union of all available pointer devices. Emilio is working on a prerequisite work for 'any-pointer' and 'any-hover' in bug 1481961. I am started working on this with dynamic pointer changes, unfortunately it won't work on GTK+ since GdkDeviceManager doesn't work as expected and it will work on MacOSX since the patch uses LookAndFeel, thus can be managed two preference values (the primary device and the union value for all devices). Anyway thanks for updating the patch!

I agree that Emilio's patch should allow us to get the any-* functions working more correctly outside of the boolean context. If things stall here until after I'm back from PTO, I'll try to help push this over the finish line again. As for the primary pointer/hover values, I've modeled those decisions on Chrome's simple choices: https://github.com/chromium/chromium/search?q=GetPrimaryPointerType&unscoped_q=GetPrimaryPointerType I also get the suspicion that they're simply querying the values over and over for the dynamic case: https://github.com/chromium/chromium/search?q=mediavaluesdynamic&unscoped_q=mediavaluesdynamic (noting how MediaValuesDynamic and CreateDynamicIfFrameExists seem to be used).

(In reply to twisniewski from comment #41) > I also get the suspicion that they're simply querying the values over and > over for the dynamic case: > https://github.com/chromium/chromium/ > search?q=mediavaluesdynamic&unscoped_q=mediavaluesdynamic (noting how > MediaValuesDynamic and CreateDynamicIfFrameExists seem to be used). They observe dynamic changes, for example https://bugs.chromium.org/p/chromium/issues/detail?id=442418 is for Android.

Hmm that bug was for all platforms (other than MacOSX).

Mostly done other than any-pointer and any-hover; https://treeherder.mozilla.org/#/jobs?repo=try&revision=4e985495a3845f998a7d784a88760796d7669427&selectedJob=193623057 I have to check whether dynamic changes work fine on Windows. I will probably fix any-pointer and any-hover in a later bug.

I filed three bugs that I will tackle later. For any-hover and any-pointer: bug 1483111 For MacOSX backend: bug 1483114 For dynamic changes on GTK+: bug 1483115

I believe this is P2.

https://drafts.csswg.org/mediaqueries-4/#hover https://drafts.csswg.org/mediaqueries-4/#pointer In this patch series, we don't introduce any-hover and any-pointer media features yet because of bug 1481961, but functionalities for them on each platform backends will be introduced in this patch series, so eIntID_AllPointerCapabilities and relevant stuff is added in this patch for the convenience that each platform backends can be reviewed at once.

Depends on D3296

Note that GDK_SOURCE_TOUCHSCREEN has been available since gtk+-3.4, GDK_SOURCE_TRACKPOINT and GDK_SOURCE_TABLET_PAD have been available since gtk+-3.22. https://developer.gnome.org/gdk3/stable/GdkDevice.html#GdkInputSource Depends on D3297

Depends on D3298

Note that the logic of IsTabletDevice() and relevant stuff are mostly a mimic of IsTabletDevice [1] and IsDeviceUsedAsATablet [2] in chromium. As for the primary pointer's capabilities, there seems no API to tell which device is the primary, so here we determine it as following priorities; 1) if the machine is in tablet mode, the tablet (touch screen) is the primary 2) if the machine has a mouse, the mouse is the primary 3) if the machine has a digitizer, it's the primary This priority is also the same as what chromium does in GetPrimaryPointerType [2] and GetAvailablePointerTypes [3]. [1] https://chromium.googlesource.com/chromium/src/+/d4c8983688edd2725a5bfb4a9538047f7a2efa27/base/win/win_util.cc#450 [2] https://chromium.googlesource.com/chromium/src/+/d4c8983688edd2725a5bfb4a9538047f7a2efa27/base/win/win_util.cc#468 [3] https://chromium.googlesource.com/chromium/src/+/d4c8983688edd2725a5bfb4a9538047f7a2efa27/ui/base/touch/touch_device_win.cc#37 [4] https://chromium.googlesource.com/chromium/src/+/d4c8983688edd2725a5bfb4a9538047f7a2efa27/ui/base/touch/touch_device_win.cc#82 Depends on D3299

Karl, moz-phab doesn't allow empty reviewers but phabricator itself allows it. Could you please review patches in comment 48 and 49?

I lost NI request. :/ Karl ^

Note that this dynamic change will not work without a patch for bug 1478576 if user doesn't touch browser windows. Depends on D3300

There is no API that we can tell which input device is the primary pointer device on Android, so we assume that the first pointer-type input device in the list of InputDevice.getDevice is the primary one. Test results on L-01F (a variant of LG-G2): - Normal boot The touchscreen is the primary device - Plugged a mouse The touchscreen is the primary device - Boot with a plugged mouse The mouse is the primary device - Plugged off the mouse The touchscreen is now the primary device - Plugged the mouse again The touchscreen is the primary device Classification in getPointerCapablities refered to chrome impelementation [1]. In addition to the classification, we add SOURCE_JOYSTICK as a 'COARSE' and 'HOVER_CAPABLE' device. [1] https://chromium.googlesource.com/chromium/src/+/d4c8983688edd2725a5bfb4a9538047f7a2efa27/ui/android/java/src/org/chromium/ui/base/TouchDevice.java#72 Depends on D3301

Though we can't write automation tests for this, I have confirmed that the input device added/removed are properly notified to 'browser.xul'. Unfortunately the notifications don't reach to conent documents because of bug 1478212, but I've also confirmed they did reach to content documents with replacing MediaFeatureValuesChanged with MediaFeatureValuesChangedAllDocuments in nsPresContext::RefreshSystemMetrics(). Depends on D3302

Comment on attachment 8999852 [details] Bug 1035774 - Factor out a function getting the windows power platform role. r=masayuki Masayuki Nakano [:masayuki] (JST, +0900) has approved the revision.

Comment on attachment 8999854 [details] Bug 1035774 - Introduce functions to get the capabilities for the primary pointer and for all available pointers on Windows. r=masayuki Masayuki Nakano [:masayuki] (JST, +0900) has approved the revision.

(In reply to Hiroyuki Ikezoe (:hiro) (PTO on Aug 20) from comment #59) > https://phabricator.services.mozilla.com/D3301#inline-11743 Gah, I did push Enter key before I left comment. :/ Jimm, could you please check the place where we need to pull nsWindow.cpp away from unified build? The patch uses GUID_DEVINTERFACE_MOUSE, thus INITGUID is needed, then unified build doesn't work for some reasons (I don't know the exact reason).

Comment on attachment 8999840 [details] Bug 1035774 - Add media feature keys for hover and pointer. r=heycam Emilio Cobos Álvarez (:emilio) has approved the revision.

(In reply to Hiroyuki Ikezoe (:hiro) (PTO on Aug 20) from comment #60) > (In reply to Hiroyuki Ikezoe (:hiro) (PTO on Aug 20) from comment #59) > > https://phabricator.services.mozilla.com/D3301#inline-11743 > > Gah, I did push Enter key before I left comment. :/ > > Jimm, could you please check the place where we need to pull nsWindow.cpp > away from unified build? The patch uses GUID_DEVINTERFACE_MOUSE, thus > INITGUID is needed, then unified build doesn't work for some reasons (I > don't know the exact reason). Not sure what you're asking here. Please clarify. If you need INITGUID in a large file like nsWindow.cpp, might be better to drop[ this code into a stand alone cpp and use it there.

Comment on attachment 8999857 [details] Bug 1035774 - Introduce functions to get the capabilities for the primary pointer and for all available pointers on Android. r=jchen Jim Chen [:jchen] [:darchons] has approved the revision.

(In reply to Jim Mathies [:jimm] from comment #62) > (In reply to Hiroyuki Ikezoe (:hiro) (PTO on Aug 20) from comment #60) > > (In reply to Hiroyuki Ikezoe (:hiro) (PTO on Aug 20) from comment #59) > > > https://phabricator.services.mozilla.com/D3301#inline-11743 > > > > Gah, I did push Enter key before I left comment. :/ > > > > Jimm, could you please check the place where we need to pull nsWindow.cpp > > away from unified build? The patch uses GUID_DEVINTERFACE_MOUSE, thus > > INITGUID is needed, then unified build doesn't work for some reasons (I > > don't know the exact reason). > > Not sure what you're asking here. Please clarify. If you need INITGUID in a > large file like nsWindow.cpp, might be better to drop[ this code into a > stand alone cpp and use it there. That makes sense. Splitt out a new file for it, and finished updating the patch now. https://phabricator.services.mozilla.com/D3301

Comment on attachment 8999858 [details] Bug 1035774 - Support dynamic input device changes for Android. r=jchen Jim Chen [:jchen] [:darchons] has approved the revision.

:jimm, could you please review attachment 8999856 [details]?

Comment on attachment 8999848 [details] Bug 1035774 - Rename WidgetUtilsGtk.{cpp,h} to WidgetUtilsGTK.{cpp,h}. Karl Tomlinson (:karlt) has approved the revision.

Comment on attachment 8999848 [details] Bug 1035774 - Rename WidgetUtilsGtk.{cpp,h} to WidgetUtilsGTK.{cpp,h}. Karl Tomlinson (:karlt) has requested changes to the revision.

Comment on attachment 8999848 [details] Bug 1035774 - Rename WidgetUtilsGtk.{cpp,h} to WidgetUtilsGTK.{cpp,h}. Karl Tomlinson (:karlt) has approved the revision.

Comment on attachment 8999848 [details] Bug 1035774 - Rename WidgetUtilsGtk.{cpp,h} to WidgetUtilsGTK.{cpp,h}. Karl Tomlinson (:karlt) has requested changes to the revision.

Comment on attachment 8999848 [details] Bug 1035774 - Rename WidgetUtilsGtk.{cpp,h} to WidgetUtilsGTK.{cpp,h}. Karl Tomlinson (:karlt) has approved the revision.

Without this change, we can't include our own gdkdevice.h in our tree. Depends on D3297

(In reply to Hiroyuki Ikezoe (:hiro) from comment #66) > :jimm, could you please review attachment 8999856 [details]? In UnregisterNotification, I would check for null in aHandle and remove those checks from the calling point.

(In reply to Jim Mathies [:jimm] from comment #73) > (In reply to Hiroyuki Ikezoe (:hiro) from comment #66) > > :jimm, could you please review attachment 8999856 [details]? > > In UnregisterNotification, I would check for null in aHandle and remove > those checks from the calling point. Done.

Intent to implement (Preference behind which this will be implemented:: None.): https://groups.google.com/forum/#!topic/mozilla.dev.platform/WhXB19qGVkI

I've decided to defer the implementation for GTK in bug 1483115 since it requires X11 functionalities, and it will take more time than I thought.

Drop NI to jimm since I've asked :aklotz for the review.

Comment on attachment 8999856 [details] Bug 1035774 - Support dynamic input device changes for windows. r=masayuki Aaron Klotz [:aklotz] has approved the revision.

Pushed by hikezoe@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/035003a0c43d Add media feature keys for hover and pointer. r=heycam,emilio https://hg.mozilla.org/integration/mozilla-inbound/rev/4124937f8ef9 Factor out a function getting the windows power platform role. r=masayuki https://hg.mozilla.org/integration/mozilla-inbound/rev/7d16ad1fa81e Introduce functions to get the capabilities for the primary pointer and for all available pointers on Windows. r=masayuki https://hg.mozilla.org/integration/mozilla-inbound/rev/3db29a826410 Support dynamic input device changes for windows. r=aklotz https://hg.mozilla.org/integration/mozilla-inbound/rev/ad515d6f27b3 Introduce functions to get the capabilities for the primary pointer and for all available pointers on Android. r=jchen https://hg.mozilla.org/integration/mozilla-inbound/rev/ff2c8e49bfb4 Support dynamic input device changes for Android. r=jchen

https://hg.mozilla.org/mozilla-central/rev/035003a0c43d https://hg.mozilla.org/mozilla-central/rev/4124937f8ef9 https://hg.mozilla.org/mozilla-central/rev/7d16ad1fa81e https://hg.mozilla.org/mozilla-central/rev/3db29a826410 https://hg.mozilla.org/mozilla-central/rev/ad515d6f27b3 https://hg.mozilla.org/mozilla-central/rev/ff2c8e49bfb4

I believe this got documented: https://developer.mozilla.org/en-US/docs/Web/CSS/Media_Queries/Using_media_queries https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/64#CSS Let me know if you think the docs need anything else. Thanks!