Filing this bug to ensure we have the means to track the searches from the following locations with separate codes:* Searches from the search bar* Searches from Firefox Start* Searches from the URL barI assume the search bar + FF Start already have separate codes, we need to make sure we can identify the searches from the URL bar with the recent changes in FF4.There might be a Google component here too (e.g. do we need to tell them about the new code?), but I'll leave that to the CCed parties to clarify.This probably also needs to block the final release of Firefox 4.

Requesting blocking for Firefox 4 final release.

For context on the URL bar searches, see bug 586821.

Just to clarify, we're talking about Google channels only, correct?

Yes, I assume our Yandex volume isn't substantial enough yet to need this (and would likely result in similar distribution anyway)

(In reply to comment #0) > Filing this bug to ensure we have the means to track the searches from the > following locations with separate codes You mean "so that Google has the means to track them", right? We don't proxy searches through URLs that we control - they go directly to Google. We use the same URL for keyword.URL and the search bar. We also use that URL for the search bar on about:home, but add the "&source=hp&channel=np" parameters to it (per bug 594675).

Given keyword.URL a unique parameter would be doable, if desired (though we'd probably need to use a hacky approach similar to the one used for about:home...). Do you want to morph this bug to cover that?

(In reply to comment #5) > You mean "so that Google has the means to track them", right? We don't proxy > searches through URLs that we control - they go directly to Google. Yes, whatever we add to the query string to separate Start page searches from search bar searches should also be added to the URL bar search. I have no idea how this is implemented, just wanted a bug on it, so it is tracked somewhere. :)

Most likely the parameter will be "channel", but I'll need to coordinate with some folks here and at Google first. I think the same approach that we've looked at for the local home page applies, and adding the location bar searches should be the only other search surface that'd be affected. Specifics to follow, but typically look to separate out queries form the home page, queries from the search bar, and queries from the location bar. The parameter will vary by provider, so it should be flexible. I'll update this bug with specifics when I have it.

We already separate out about:home from search bar (see comment 5 / bug bug 594675). Current state of affairs is this: 1) URL bar keyword search - Normal Google URL 2) Search bar search - Normal Google URL 3) about:home search field - Normal Google URL, plus "source=hp&channel=np" 4) Selected text context menu search - Normal Google URL Adding a parameter for 1) is easy, just need to know what parameter to add (channel=kw?). Adding a parameter for 4) would involve more work, so I'd like to avoid it. (It's likely a tiny number compared to the others...)

(In reply to comment #9) > We already separate out about:home from search bar (see comment 5 / bug bug > 594675). > > Current state of affairs is this: > 1) URL bar keyword search > - Normal Google URL > 2) Search bar search > - Normal Google URL > 3) about:home search field > - Normal Google URL, plus "source=hp&channel=np" > 4) Selected text context menu search > - Normal Google URL > > Adding a parameter for 1) is easy, just need to know what parameter to add > (channel=kw?). Works for me. > Adding a parameter for 4) would involve more work, so I'd like to avoid it. > (It's likely a tiny number compared to the others...) Yeah, I wouldn't worry about that one for now.

Right now, the comments here lead me to believe that we're just talking about making google know where a search came from. I don't see us having anything that tells us, aka a feedback experiment, a search came from.

Kev: you want to take this to Steve? I suspect they'll want to modify the channel= variable, as you say. They're using "np" for "new page"; perhaps "lb" for "location bar"? :)

Not sure I understand why this is not blocking; having the ability to track how many of our users prefer the URL bar searches over the search box seems to be very important data to me, and is also important when considering the evolution and possible merge of search field & URL input fields in the future. Re-requesting blocking.

Limi, what you're asking for doesn't do what you expect it to, AFAICT. With what folks are talking about in this bug, google will know how our folks use url bar searches, we will not. Is that your intention? Also, http://mxr.mozilla.org/mozilla-central/source/browser/app/profile/firefox.js#219 sets keyword.URL to blank now.

keyword.URL isn't relevant, we can make location bar searches differentiable (to Google, not to us, as you point out) using the same technique we used for Yandex (x-moz-keywordsearch).

Blocking- for comment #14 and #15. IIUC, this change doesn't do what Alex thinks it does, and outside of that, there's a better way to do it.

I explained to Limi how the code works, so he now has an understanding for why keyword.URL is irrelevant. Changed the bug title to reflect this.

(In reply to comment #16) > Blocking- for comment #14 and #15. IIUC, this change doesn't do what Alex > thinks it does, and outside of that, there's a better way to do it. I was asked by Jim to make sure we track this on the Google side. He can get the data on how search is initiated, and will — just because we currently don't do it, doesn't mean we don't want to. Happy to discuss it if you need more data, or more clarification. Re-requesting blocking.

And, carrying over information from bug 624054: Ideally, searches from the Location Bar should have a source attribute, I propose we add "&source=lb" for searches done here.

We'll need to confirm with Google on what an acceptable parameter would be, as they'll be pulling it. I believe they have a field for channels, but can't remember off-hand what it is. I'll talk to them tomorrow and put a patch in when I have it, but generally it can't be an arbitrary attribute. While the current implementation of application/x-moz-keywordsearch is irrelevant in this use-case, I would like to see if we can't make it relevant for all other search surfaces in the future; it would help simplify search plugin integration/admin.

(In reply to comment #20) > And, carrying over information from bug 624054: > > Ideally, searches from the Location Bar should have a source attribute, I > propose we add "&source=lb" for searches done here. (In reply to comment #21) > We'll need to confirm with Google on what an acceptable parameter would be, as > they'll be pulling it. I believe they have a field for channels, but can't > remember off-hand what it is. I'll talk to them tomorrow and put a patch in > when I have it, but generally it can't be an arbitrary attribute. There is a parameter called "channel" that is used in some cases, but I don't know what it means. Setting the "source" parameter seems to make sense, because it's currently set to "hp" (short for home page) when searching from about:home.

Per above, it's something we shouldn't assume. Beltzner: could you confirm Google asked us to use "source" for the local start page? I'll use that as the lead-in with Google if so, and advise them of plans if we're all in agreement. > There is a parameter called "channel" that is used in some cases, but I don't > know what it means. Setting the "source" parameter seems to make sense, because > it's currently set to "hp" (short for home page) when searching from > about:home.

Hey all...jumping in here late. Sorry for the confusion by not being on here earlier. Overarching Goal of this request 1) Ensure we start getting reports from Google that separates Location Bar Searches (lb) from Startpage Searches vs Chrome Searches. We get the latter 2 already on a monthly basis from Google in a critical report we analyze deeply. 2) Without this separation of Location Bar vs Chrome, we will lose some critical data on our User Behavior for which we can build future great UI/product offerings In reply to: (comments 11 and 14: Axel) - Yes, Google will know but they already know whether or not we do anything (they have enough statisical data on their own Chrome product feature). By adding the attribute to FF4, we can force Google to report it back to us at Mozilla (comment 13: Limi) - I agree with this comment (comments 15 & 16 - Gavin & Dietrich) - sounds like there might be a better way of doing it? That's great. Don't really care about the way as long as the requirement is met that Google starts adding this Location Bar Field to our Monthly Report they deliver to us. (comment 21 - Kev) - Yes, we definitely need to communicate with Google to ensure they are aware of any change and can and will start reporting it back to us in a separate field

Bug 594675 is where we added the about:home parameter. Not sure I understand the previous comments about x-moz-keywordsearch - we do want to use it for this, just like we did with Yandex: http://hg.mozilla.org/l10n-central/ru/rev/07ae2c762256

(In reply to comment #25) > Bug 594675 is where we added the about:home parameter. > > Not sure I understand the previous comments about x-moz-keywordsearch - we do > want to use it for this, just like we did with Yandex: > http://hg.mozilla.org/l10n-central/ru/rev/07ae2c762256 my age-addled brain. I thought we had removed that bit. my bad.

Unowned blocker, over to Frank.

I'm taking this off the blocker list. There's nothing stopping us from doing this after we ship Firefox 4, and a precursor step of working out the appropriate codes with Google still hasn't happened.

Unassigning self, unless it becomes a blocker again or until we work out the appropriate codes with Google. I'm happy to pick it up later though, if no one else is.

OK, finally got confirmation from Google on what to do here: The query string for the location bar searches should be: http://www.google.com/search?ie=UTF-8&oe=UTF-8&sourceid=navclient&client=google&channel=fflb&q= Google representative said: "The only 2 new parameters being added to the existing setting are client=google and channel=fflb."

This adds the two extra parameters for URL-bar triggered searches. With this patch, the URLs used for a query of "test 123" are: Search bar (unchanged): http://www.google.com/search?q=test+123&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a URL bar (two added parameters): http://www.google.com/search?q=test+123&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a&client=google&channel=fflb

There's a problem with comment 30 that I failed to notice earlier - we already send a "client" parameter, whose value is typically "firefox-a" (or "firefox" for builds where Google is not the default engine). I imagine we'll just want to continue sending that, and not add the additional client=google.

Gavin: correct. client codes need to match what we're sending in the search bar.

I asked Kathy from Google, and she said: > For tracking of the awesome bar, you will need to set the client=google (not firefox-a or firefox in this use case). Please note that tracking will not work otherwise and all our previous testing was using this new keyword.URL setting, with client=google and channel=fflb. This setting is only for searches coming from the awesome bar. Kev, can you make sure we're doing the right thing here with Google?

Bleh, damn Bugzilla. Let's make that readable: “For tracking of the awesome bar, you will need to set the client=google (not firefox-a or firefox in this use case). Please note that tracking will not work otherwise and all our previous testing was using this new keyword.URL setting, with client=google and channel=fflb. This setting is only for searches coming from the awesome bar.”

That doesn't sound right - we still want these searches to be identified as coming from Firefox. Changing the value of the "client" parameter doesn't really make sense, assuming that's what it's used for.

note to self: need to also update browser_keywordSearch.js

We still need to get confirmation about the parameters, but no matter what the details end up being, this is the approach we'll need. I still think a simple additional parameter is correct.

Comment on attachment 527409 [details] [diff] [review] real patch with test fix r+ assuming you check with Google to confirm the arg is what they'll want.

We're still in a holding pattern with Google, so please hold off landing this. Hope to have red light/green lighr next week, per Google reps.

Comment on attachment 527409 [details] [diff] [review] real patch with test fix This works for the moment, but I think we'd rather wait to fix bug 587780 and use that mechanism.

This was fixed by bug 724116. Bug 587780 will eventually make it a little more maintainable in general.