Currently the tour is often triggered by pages that aren't articles, and aren't suited to be viewed with Reader View. Having this tour appear out of context greatly reduces it's chances of helping the user and as the tour will only appear once, we wasted our chance of telling the user about a very handy feature. Reader View is currently triggered on sites like YouTube and Pinterest which are very popular sites which many users might browse quite soon after installing Firefox which greatly increases the chance of the tour not being successful. I understand we trigger the tour on the first page that is detected as potential article. As this detection has many false positives we should consider additional measures to increase the chance of this tour being shown on an actual article. Such measures could be a blacklist, excluding YouTube, Pinterest and potentially other popular sites from triggering the tour. Or a whitelist, listing sites we know are good candidates for the tour. (probably way harder)

I am pretty uneasy about this. Having a list of sites where we *will* show the tour risks the user never seeing the tour. Having a list of sites where we refuse to show the tour is likely to get outdated as the internet changes, and/or be ineffective depending on people's surfing habits. Just calling out youtube and pinterest (which is what the deps do) sounds anecdotal and reactive. If we were going the route of having some kind of blocklist, we should have a better idea of where we show this when we're "not supposed to", but collecting that data via telemetry is not OK because privacy. The next best thing would be to manually go through something like the Alexa top 100 sites or whatever, but even that seems like it is unlikely to be particularly effective for our purposes because the "long tail of the internet" is, well, long. What constitutes "tour success" and what are we aiming for here? We aren't currently investing in reader mode, and if we were, this still wouldn't be where I would aim energy. I'd prefer to just make the detection and/or rendering better. If we think the tour is distracting and unsuccessful at telling people about reader mode, I think we should just remove the tour altogether. Thoughts? Am I missing the point completely here?

I don't like the idea of having different thresholds or lists as it adds complexity and will never be perfect. Are we fine with the button appearing on YouTube but just not promoting it on those pages? IMO if we change the panel to not be sticky (remove noautohide) then I think this is less of an issue.

Having an accurate detection would of course be the best solution. But I guess that would be the most effort. With Reader View we built a great feature which might be helpful to many people, but with triggering the tour on non-article pages we lose the one way that might help people discover the feature, or worse, we confuse or annoy them with irrelevant or broken functionality. (In reply to :Gijs Kruitbosch from comment #1) > I am pretty uneasy about this. Having a list of sites where we *will* show > the tour risks the user never seeing the tour. Having a list of sites where > we refuse to show the tour is likely to get outdated as the internet > changes, and/or be ineffective depending on people's surfing habits. Never is probably better then on a wrong page, as we then at least do not confuse the user, but would drastically reduce exposure of Reader View or result in a very long list. A blocklist however will never be ineffective across all users, but at least increase the chance of the tour being triggered on an actual article which is what I would prefer us doing. (Only if this is a small/medium amount of effort.) Even if it is not 100% accurate, it will be more accurate as we are now. (And I would volunteer proposing such a list if there is a real chance of us implementing it.) > Just calling out youtube and pinterest (which is what the deps do) sounds > anecdotal and reactive. If we were going the route of having some kind of > blocklist, we should have a better idea of where we show this when we're > "not supposed to", but collecting that data via telemetry is not OK because > privacy. The next best thing would be to manually go through something like > the Alexa top 100 sites or whatever, but even that seems like it is unlikely > to be particularly effective for our purposes because the "long tail of the > internet" is, well, long. Even with only checking the Alexa top 50 sites we might drastically increase our accuracy due to the quick drop off in monthly visits. http://www.theguardian.com/technology/blog/2010/jul/06/graph-top-1000-websites-long-tail-internet > What constitutes "tour success" and what are we aiming for here? We aren't > currently investing in reader mode, and if we were, this still wouldn't be > where I would aim energy. I'd prefer to just make the detection and/or > rendering better. If we think the tour is distracting and unsuccessful at > telling people about reader mode, I think we should just remove the tour > altogether. Removing the tour altogether would result in Reader View getting even less exposure and probably lead into the same path that panorama took. (Hide it > wonder why nobody uses it > remove it) Which I would rather not see as the feature is pretty helpful, but without giving people a chance to discover it we can not measure how useful it really is. > Thoughts? Am I missing the point completely here? No, you did not miss the point. Currently the tour is in a weird place. Not dead, but not too effective either. If we remove the tour we should consider making the feature into an add-on instead of letting it hang around Firefox. This might give us other ways to promote it and provide us with better data on how many people actually care about it. (In reply to Matthew N. [:MattN] from comment #2) > I don't like the idea of having different thresholds or lists as it adds > complexity and will never be perfect. Are we fine with the button appearing > on YouTube but just not promoting it on those pages? I don't think we need perfect. I an internet as huge and fluent as it is, we can never be perfect. If users got the chance to understand what the button does (aka. having seen the tour on an actual article.), they might be less motivated to click it on a non article, as they already learned that it has to do with articles; and better understand why it does not work if they still click it on a not-article page. > IMO if we change the panel to not be sticky (remove noautohide) then I think > this is less of an issue. This would come close to removing the tour, as it would reduce exposure. (see my thoughts on removing the tour.)

I talked to Michael Verdi about this bug today, and I think it's interesting: * we should be able to agree intuitively that engaging the read mode *tour* on youtube.com and pinterest.com is at best confusing, maybe even user-hostile * it's also apparent to me that we don't really understand the size of this problem or how it's affecting Firefox users, we instead have compelling user research showing confusion when users do run into the situation What I'd like to see is the following: * let's blacklist these two domains in particular * let's add telemetry to count two things: 1. how often we would have showed the reader mode tour to users on youtube and pinterest 2. how often we showed the reader mode tour This should give us an idea of impact, and also mitigate the immediate issue for users. I would much prefer for new users that we offer them the reader mode tour on a page more suited to reader mode.

(In reply to Jeff Griffiths (:canuckistani) (:⚡︎) from comment #4) > What I'd like to see is the following: > > * let's blacklist these two domains in particular Why does this ignore pretty much all of comment 1 through 3, all of which agree that this is not a good idea ? (In reply to Markus Jaritz [:maritz] (UX) from comment #3) > (In reply to :Gijs Kruitbosch from comment #1) > > What constitutes "tour success" and what are we aiming for here? We aren't > > currently investing in reader mode, and if we were, this still wouldn't be > > where I would aim energy. I'd prefer to just make the detection and/or > > rendering better. If we think the tour is distracting and unsuccessful at > > telling people about reader mode, I think we should just remove the tour > > altogether. > > Removing the tour altogether would result in Reader View getting even less > exposure and probably lead into the same path that panorama took. (Hide it > > wonder why nobody uses it > remove it). There is primary UI for the feature with the tour gone, so I strongly disagree that this is what removing the tour means. If things with no tour have no exposure then we should be removing almost everything in the URL bar, including but not limited to the identity block, the popup blocker, mixed content blocking, and a number of other things. That is clearly not a good idea. What's more, that is also not actually the "path that panorama took" - panorama was removed because of the immense technical debt and it blocking our other work, like e10s. We could afford to remove it rather than fix it because usage was low, but that was not a *reason* in itself. Reader Mode does not have any of these problems - neither low usage, nor immense technical debt, nor does it actively mess with other features. > If we remove the tour we should consider making the feature into an add-on > instead of letting it hang around Firefox. This might give us other ways to > promote it and provide us with better data on how many people actually care > about it. As I said earlier, if we want to invest in this, I would prefer to invest in reader mode itself, which will also fix the tour. So far I have not seen bugs filed for either the youtube or the pinterest case. > (In reply to Matthew N. [:MattN] from comment #2) > > IMO if we change the panel to not be sticky (remove noautohide) then I think > > this is less of an issue. > > This would come close to removing the tour, as it would reduce exposure. > (see my thoughts on removing the tour.) I really don't see why. It would still pop up, but people could close it more easily. Why does that "reduce exposure" ? The one other thing that we could do here that stops short of actually investing in reader mode but would maybe help with concerns in this bug, is require the tour to have a working reader mode for the page in question (ie not the "oh, it didn't work" thing after clicking the button). I don't know for certain if that will help (it could be that we just show a tiny bit of content in reader mode, instead of something useful?), because nobody here has actually filed bugs about the broken cases (despite me having asked them a week ago...) and indicated what currently happens, and I can't reproduce them myself.

I don't think a blacklist is that big of a deal -- we did talk about doing it when originally implementing readerview, and it's a cheap way to fix "site X isn't working/appropriate in readerview" without requiring someone to dive into the complexity of RV's heuristics. (And risk breaking desired sites when we change it.) Yes, it's a hack, and the "right" fix would be to fix the heuristics or readability. But it's a cheap and simple hack, which seems appropriate for the low investment we've made in these since shipping. [I do think we should keep things simple, though. If a page passes heuristics + blacklist, it should be eligible for the triggering the tour. I don't think it's worthwhile to add a separate layer of checks just for the tour.] (In reply to Jeff Griffiths (:canuckistani) (:⚡︎) from comment #4) > * let's add telemetry to count two things: > 1. how often we would have showed the reader mode tour to users on youtube > and pinterest > 2. how often we showed the reader mode tour I think the first is going to have privacy issues. It would be more straightforward to just fix the issue. I'm not sure what you mean by the second. I would expect basically every new user to see the tour soon after starting to use Firefox. Aside: another simple way to fix this issue is to just remove the tour. It's already served its primary purpose (introducing existing Firefox users to a new feature), and its not such a major feature that we _need_ to promote it to new users. We spent time at the recent onboarding summit talking about what a mess our onboarding flow is, and the cornucopia of introductions we toss in front of new users feels like more of a net-annoyance. In fact, I'd really rather we just killed the tour as the preferred fix. Simple to do, and takes care of multiple issues.

(In reply to Justin Dolske [:Dolske] from comment #6) ... > (In reply to Jeff Griffiths (:canuckistani) (:⚡︎) from comment #4) > > > * let's add telemetry to count two things: > > 1. how often we would have showed the reader mode tour to users on youtube > > and pinterest > > 2. how often we showed the reader mode tour > > I think the first is going to have privacy issues. It would be more > straightforward to just fix the issue. Okay. > I'm not sure what you mean by the second. I would expect basically every new > user to see the tour soon after starting to use Firefox. It helps us understand the first number, eg knowing the number of attempts to show the tour on youtube and pinterest is much more useful if you also know the total number of times the tour is actually shown. It helps provide size to the debate. > Aside: another simple way to fix this issue is to just remove the tour. It's > already served its primary purpose (introducing existing Firefox users to a > new feature), and its not such a major feature that we _need_ to promote it > to new users. We spent time at the recent onboarding summit talking about > what a mess our onboarding flow is, and the cornucopia of introductions we > toss in front of new users feels like more of a net-annoyance. > > In fact, I'd really rather we just killed the tour as the preferred fix. > Simple to do, and takes care of multiple issues. I'd like some discussion on what we're going to do instead of the tour and what the future of the feature is. I would want to know more about what the size of this problem is, which was why I was suggesting instrumentation. How many times do we actually offer the tour on youtube and pinterest? We don't know, and I'd rather know than just guess. My suggestion of the blacklist was an attempt to at least prevent a known bad experience at the same time. I'd also like to see ideas on how to promote the feature in place of this tour, if we feel Reader Mode has a future inside Firefox. I also think this is an interesting idea: > If we remove the tour we should consider making the feature into an add-on instead of letting it hang > around Firefox. This might give us other ways to promote it and provide us with better data on how many > people actually care about it.

(In reply to Jeff Griffiths (:canuckistani) (:⚡︎) from comment #7) > > In fact, I'd really rather we just killed the tour as the preferred fix. > > Simple to do, and takes care of multiple issues. > > I'd like some discussion on what we're going to do instead of the tour and > what the future of the feature is. I think those two things are orthogonal. There are plenty of Firefox features (most of them, really!) that have no associated tour. That doesn't mean they're not important. And I'm certainly not suggesting or scheming to remove reader view. :) In recent history we started adding tours for features launched in our fall/spring campaigns, but I don't think anyone's really taken a look at how effective they are. (Well, I did see data on the Hello tour, and that wasn't good.) Or, more worrysome, what their cumulative effect is on poor new-user UX... EG, launch Firefox for the first time, open a tab and go to a news site. You're bombarded with a default browser prompt, a new-tab page info overlay, a reader view promo (and somewhere in there a data choices prompt may be shown, too). We should really plan on removing tours (at least in their current form) some period of time after their introduction. Or coming up with a better, holistic experience for how we introduce Firefox features to new users and what those the most effective features to promote should be. All of this is to say that we added a tour a while back because it was a new shiny thing, and we shouldn't feel compelled to keep a tour permanently. > How > many times do we actually offer the tour on youtube and pinterest? We don't > know, and I'd rather know than just guess. My suggestion of the blacklist > was an attempt to at least prevent a known bad experience at the same time. A blacklist is something we can use to bandaid over known problematic sites, but the root issue (showing the promo on a page that's misdetected as being useful for Reader View) isn't something we can really measure, because it's an inherently fuzzy or undecidable problem.

Poking into this again… Dolske made it sound like a blacklist would be a relatively cheap (yet incomplete) solution to a very real quality issue. I don't see us investing heavily enough into reader view soon in order to fix the underlying detection mechanism, but a blacklist would help us to suck less often. So let's do that for now.

It could also help if there would be a way for websites to tell Firefox that the reading mode should not be offered, see bug 1204217.

(In reply to Sören Hentzschel from comment #10) > It could also help if there would be a way for websites to tell Firefox that > the reading mode should not be offered, see bug 1204217. In practice this would get abused by websites to "protect their content". Cf. https://github.com/mozilla/readability/issues/257 and https://github.com/mozilla/readability/issues/255 . Safari does not offer this either. I strongly feel we should not do this. (In reply to Philipp Sackl [:phlsa] (Firefox UX) please use needinfo from comment #9) > Poking into this again… > Dolske made it sound like a blacklist would be a relatively cheap (yet > incomplete) solution to a very real quality issue. > I don't see us investing heavily enough into reader view soon in order to > fix the underlying detection mechanism, but a blacklist would help us to > suck less often. > So let's do that for now. Why not remove the tour as suggested in comment #6 and comment #8? That would be even cheaper.

(In reply to Philipp Sackl [:phlsa] (Firefox UX) please use needinfo from comment #9) > Poking into this again… > Dolske made it sound like a blacklist would be a relatively cheap (yet > incomplete) solution to a very real quality issue. > I don't see us investing heavily enough into reader view soon in order to > fix the underlying detection mechanism, but a blacklist would help us to > suck less often. > So let's do that for now. Agreed.

(In reply to :Gijs Kruitbosch from comment #11) > Why not remove the tour as suggested in comment #6 and comment #8? That > would be even cheaper. Reader mode is currently a great point of differentiation from one of our main competitors, but it hard to find if you don't know that it exists. This is why the »tour« (it's really just an indication of the existence of the feature) is needed.

Review commit: https://reviewboard.mozilla.org/r/30567/diff/#index_header See other reviews: https://reviewboard.mozilla.org/r/30567/

Comment on attachment 8707069 [details] MozReview Request: Bug 1210366 - block youtube and pinterest from reader mode, r?MattN https://reviewboard.mozilla.org/r/30567/#review27355 ::: browser/modules/ReaderParent.jsm:150 (Diff revision 1) > + "youtube.com", > + "pinterest.com" > + ]; Nit: add the trailing comma so blame doesn't change if we add a site. Nit: sort the list alphabetically ::: browser/modules/ReaderParent.jsm:151 (Diff revision 1) > + "pinterest.com" This also blocks blog.pinterest.com which should get the tour IMO. I would rather be more conservative and use www.pinterest.com Youtube.com doesn't seem to host articles on other subdomains so this is fine and will also catch m.youtube.com on mobile.

https://reviewboard.mozilla.org/r/30567/#review27359 ::: browser/modules/ReaderParent.jsm:135 (Diff revision 1) > - currentUriHost && !currentUriHost.endsWith("mozilla.org")) { > + currentUriHost && this._canShowTourForHost(currentUriHost)) { Why not perform this check near the actual call to ReaderMode.isProbablyReaderable() instead? The sites we're adding here are ones where the reader view shouldn't be active on at all. So while suppressing the tour avoids calling attention to our mistake, we should really just suppress reader view entirely. (There's also the minor point that the mozilla.org tour suppression is subtly different -- we do that because it's the easiest way to suppress the tour on whatsnew/firstrun type pages, which we open automatically and would lead to a confusing experience.)

(In reply to Justin Dolske [:Dolske] from comment #16) > https://reviewboard.mozilla.org/r/30567/#review27359 > > ::: browser/modules/ReaderParent.jsm:135 > (Diff revision 1) > > - currentUriHost && !currentUriHost.endsWith("mozilla.org")) { > > + currentUriHost && this._canShowTourForHost(currentUriHost)) { > > Why not perform this check near the actual call to > ReaderMode.isProbablyReaderable() instead? > > The sites we're adding here are ones where the reader view shouldn't be > active on at all. So while suppressing the tour avoids calling attention to > our mistake, we should really just suppress reader view entirely. I thought we were blacklisting just for the tour's sake, not for reader view itself. Did I misunderstand the last few comments here? (In reply to Matthew N. [:MattN] (vacation Jan. 1 – 10) from comment #15) > Youtube.com doesn't seem to host articles on other subdomains so this is > fine and will also catch m.youtube.com on mobile. Note that this code does not run on mobile.

https://reviewboard.mozilla.org/r/30567/#review27359 > Why not perform this check near the actual call to ReaderMode.isProbablyReaderable() instead? > > The sites we're adding here are ones where the reader view shouldn't be active on at all. So while suppressing the tour avoids calling attention to our mistake, we should really just suppress reader view entirely. > > (There's also the minor point that the mozilla.org tour suppression is subtly different -- we do that because it's the easiest way to suppress the tour on whatsnew/firstrun type pages, which we open automatically and would lead to a confusing experience.) Opinions vary on whether Reader View should be enabled on www.youtube.com as videos can have significant descriptions including transcripts or lyrics and IMO there is value in a reader friendly version of that. Example: about:reader?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DoZdiXvDU4P0 which is being fixed in bug 1230050 to remove two lines which should be hidden.

(In reply to :Gijs Kruitbosch from comment #17) > > Why not perform this check near the actual call to > > ReaderMode.isProbablyReaderable() instead? > > > > The sites we're adding here are ones where the reader view shouldn't be > > active on at all. So while suppressing the tour avoids calling attention to > > our mistake, we should really just suppress reader view entirely. > > I thought we were blacklisting just for the tour's sake, not for reader view > itself. Did I misunderstand the last few comments here? Yes, comment 6: > [I do think we should keep things simple, though. If a page passes > heuristics + blacklist, it should be eligible for the triggering the tour. I > don't think it's worthwhile to add a separate layer of checks just for the > tour.] There is value in having this code suppress the reader icon entirely, so we have the possibility to use it as a simple bandaid for "reader view shouldn't be shown on Site X" and "reader view is broken on Site Y" bugs. Currently we can only fix these by having someone investigate why it's broken and figure out how to fix it without breaking something else. (Which is the ideal way to fix things, but is a lot more effort.) (In reply to Matthew N. [:MattN] from comment #18) > Opinions vary on whether Reader View should be enabled on www.youtube.com as > videos can have significant descriptions including transcripts or lyrics and I think that's a non-goal and extreme edge case. Reader View should focus on the experience for pages/sites where the primary content is article-scale text.

(In reply to Matthew N. [:MattN] from comment #15) > This also blocks blog.pinterest.com which should get the tour IMO. I would > rather be more conservative and use www.pinterest.com Pinterest uses region-specific hosts like "uk.pinterest.com" and so on, so this isn't possible.

Comment on attachment 8707069 [details] MozReview Request: Bug 1210366 - block youtube and pinterest from reader mode, r?MattN Review request updated; see interdiff: https://reviewboard.mozilla.org/r/30567/diff/1-2/

Comment on attachment 8707069 [details] MozReview Request: Bug 1210366 - block youtube and pinterest from reader mode, r?MattN https://reviewboard.mozilla.org/r/30567/#review28275 How convenient that we have a blocklist already…

[bugday-20160323] Status: RESOLVED,FIXED --> VERIFIED Component: Name Firefox Version 46.0b2 Build ID 20160314144540 Update Channel beta User Agent Mozilla/5.0 (Windows NT 6.1; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0 OS Windows 7 SP1 x86_64 Actual Results: As expected Expected Results: Reader mode is not present for sites like Youtube, Pinterest. This bug is not present for Firefox 46.0b2