Closed Bug 1525076 Opened 6 years ago Closed 4 years ago

The nsIMacAttributionService not returning the referrerURL

Categories

(Firefox :: General, defect, P2)

defect

Tracking

()

VERIFIED FIXED
83 Branch
Tracking Status
firefox66 - wontfix
firefox67 --- wontfix
firefox83 --- verified
firefox84 --- verified

People

(Reporter: pdehaan, Assigned: nalexander)

References

Details

Attachments

(8 files, 3 obsolete files)

(deleted), image/png
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/x-phabricator-request
Details
(deleted), text/plain
mmccorquodale
: data-review+
Details

Ref: https://bugzilla.mozilla.org/show_bug.cgi?id=1511071

Trying to do a complete end-to-end flow using https://gist.github.com/piatra/f0a2b9f9f700cbe19afd2ebf1561e6ee but I'm always getting an empty attributed object when I get to the following line in the steps:

AttributionCode.getAttrDataAsync().then(console.log) // something with content `{EF522540-89F5-46b9-B6FE-1829E2B572C6}`

Testing on macOS Mojave (latest) and High Sierra (latest-1). I asked Benny from my team to also test and he was getting the same results as me.

I was speaking w/ @mixedpuppy this morning and he was suggesting it may be due to a dialog prompting me when I try and run the app from my ~/Downloads/ folder, so I'm trying to verify that, but @ddurst asked me to file an issue.

Flags: needinfo?(ddurst)

Odd, I tested this in Nightly v67 (copying the config from Release v65), and I got the following error in the Browser Scratchpad:

ChromeUtils.import("resource:///modules/AttributionCode.jsm");
ChromeUtils.import("resource://gre/modules/addons/AddonRepository.jsm");

AttributionCode.getAttrDataAsync().then(console.log)

/*
Exception: ReferenceError: AttributionCode is not defined
@Scratchpad/1:4:1
getEvalResult@resource://devtools/server/actors/webconsole/eval-with-debugger.js:134:18
exports.evalWithDebugger@resource://devtools/server/actors/webconsole/eval-with-debugger.js:105:18
evaluateJS@resource://devtools/server/actors/webconsole.js:1005:22
evaluateJSAsync@resource://devtools/server/actors/webconsole.js:910:22
onPacket@resource://devtools/server/main.js:1279:15
send/<@resource://devtools/shared/transport/local-transport.js:64:11
exports.makeInfallible/<@resource://devtools/shared/ThreadSafeDevToolsUtils.js:109:14
exports.makeInfallible/<@resource://devtools/shared/ThreadSafeDevToolsUtils.js:109:14
*/

Here's a simple script to set the attribution on a downloaded app. https://pastebin.com/kWZ3DhdZ

Starting from a shell[1] everything works, starting from Finder (or using "open Firefox.app" in shell) it doesn't.

[1] path/to/Firefox.app/Contents/MacOS/firefox-bin -P profile

just want to confirm - that the failure on OSX is a graceful one. So if source attribution isn't returning information on OSX, the user just sees the default onboarding experience. That would be a silent failure - nothing user visible.

(In reply to :shell escalante from comment #3)

just want to confirm - that the failure on OSX is a graceful one. So if source attribution isn't returning information on OSX, the user just sees the default onboarding experience. That would be a silent failure - nothing user visible.

A very graceful fiery ball is flung through the screen at the user. It's basically silent, things will continue working as there is nothing to see.

OK, I think I got a bit closer... I downloaded Firefox 65 and ran the new script (and verified the attributes were copied from the new script):

$ ./showattrs.sh feb-03/Firefox65.app
$ ./showattrs_old.sh feb-03/Firefox65.app
054B329B-8F38-473A-8DD9-7AEFFA5DD2E8|570995305.0|org.mozilla.nightly|Nightly|http://dummy.com/file.zip|||0||https://www.mozilla.org/en-US/firefox/download/thanks/?utm_campaign=non-fx-button&utm_content=%7BEF522540-89F5-46b9-B6FE-1829E2B572C6%7D&utm_medium=referral&utm_source=addons.mozilla.org|

Running the following in the Scratchpad (Browser context) gives me some output:

ChromeUtils.import("resource:///modules/AttributionCode.jsm");
ChromeUtils.import("resource://gre/modules/addons/AddonRepository.jsm");

AttributionCode.getAttrDataAsync().then(console.log)

/*
Object {
  source: "addons.mozilla.org",
  medium: "referral",
  campaign: "non-fx-button",
  content: "{EF522540-89F5-46b9-B6FE-1829E2B572C6}"
}
*/

I'm still not seeing any add-ons preinstalled in about:addons.

I'll test in Firefox 66 and see if I get the same results.

(In reply to Peter deHaan [:pdehaan] from comment #5)

Running the following in the Scratchpad (Browser context) gives me some output:

Ok, you're getting the attribution data there.

How are you starting firefox when you get data?

I'm still not seeing any add-ons preinstalled in about:addons.

That would be a different problem than this bug.

Same results w/ FF66 (DevEdition). See attached screenshot.

I'm running the FF65 and FF66 via the CLI, like you suggested:

$ ./feb-03/Firefox66.app/Contents/MacOS/firefox-bin -P profile66

Where that seems to display the profile manager at launch, then I create a new profile and then check about:addons.

(In reply to Peter deHaan [:pdehaan] from comment #8)

I'm running the FF65 and FF66 via the CLI, like you suggested:

Ok. That is expected to work at this point. This bug is basically that it doesn't work if you start normally as a user would.

I've been trying to reproduce this with a local build, but I keep getting "Exception: ReferenceError: AttributionCode is not defined" no matter how I start Firefox. Is there anything I'm missing to get this working with a local build? I would assume that AttributionCode would always be defined, even if no attributes are present.

Flags: needinfo?(mixedpuppy)
Blocks: 1468680

(In reply to Stephen A Pohl [:spohl] from comment #10)

I've been trying to reproduce this with a local build, but I keep getting "Exception: ReferenceError: AttributionCode is not defined" no matter how I start Firefox. Is there anything I'm missing to get this working with a local build? I would assume that AttributionCode would always be defined, even if no attributes are present.

I've no idea on this. Are you testing a scratchpad script? Is it in browser environment?

Flags: needinfo?(mixedpuppy)

(In reply to Shane Caraveo (:mixedpuppy) from comment #11)

(In reply to Stephen A Pohl [:spohl] from comment #10)

I've been trying to reproduce this with a local build, but I keep getting "Exception: ReferenceError: AttributionCode is not defined" no matter how I start Firefox. Is there anything I'm missing to get this working with a local build? I would assume that AttributionCode would always be defined, even if no attributes are present.

I've no idea on this. Are you testing a scratchpad script? Is it in browser environment?

Yes (scratchpad script from comment 5) and yes (browser context). I was going to look into this to figure out what might be going on here, but at this point I can't even get to the point outlined in comment 5.

From David's comments in Trello it sounds to me like this may not block the feature rollout (even to Mac users). So, the Return to AMO feature may not work for Mac users but will not do anything negative, either, is that correct?

(In reply to Liz Henry (:lizzard) (use needinfo) from comment #13)

From David's comments in Trello it sounds to me like this may not block the feature rollout (even to Mac users). So, the Return to AMO feature may not work for Mac users but will not do anything negative, either, is that correct?

correct.

I have a WIP E2E test plan at https://github.com/pdehaan/return-to-amo-e2e-test which covers downloading Firefox Nightly (v67) and using mixedpuppy's https://pastebin.com/kWZ3DhdZ script to copy attributes and then launch Firefox Nightly. I'm not seeing the about:welcome page open in the new profile, but if I open about:welcome in a new tab it will prompt me to install the SearchPreview add-on. :+1:

No need for me to track this if it isn't blocking the feature release and has no negative effect on users.

Flags: needinfo?(ddurst)

QA notes that this issue still blocks some of their end-to-end testing.

(In reply to Liz Henry (:lizzard) (use needinfo) from comment #17)

QA notes that this issue still blocks some of their end-to-end testing.

Why are they testing OSX at this point?

Flags: needinfo?(lhenry)

Ah, maybe this one isn't relevant and they were referring to bug 1511071, which is about testing Windows.

Flags: needinfo?(lhenry)
Component: Activity Streams: Newtab → Installer

Bug 1344771 landed in ::General, so moving there ?

Component: Installer → General
Depends on: 1344771

Can somebody assign a priority to this bug please?

David and Shane, there still seems to be confusion here. If this isn't relevant, should we close the bug, give it a P5 or something, or can you explain further to QA that they don't need to test on MacOS? That was my understanding.

Flags: needinfo?(ddurst)
Priority: -- → P2

This is not a blocker for RTAMO and there is no plan to fix it for 66/67. We do want to fix it if possible.

Flags: needinfo?(ddurst)

I finally got a chance to dig into this. The "console vs. GUI" discrepancy is spurious: what is really happening is that launching via the GUI requires the user to override GateKeeper by going to Security & Privacy > Open Anyway. Doing that updates the GateKeeper flags, and then the resulting quarantine data is not returned by CFURLCopyResourcePropertyForKey. That is, both the GUI App (immediately) or a console App (launched subsequently) will find the quarantine data. I assume that this is just the way that the macOS API is: once the GK flags reach some state, quarantine information is not returned. Moons ago, somebody else on the internet witnessed the same thing.

Now, what is to be done? Under the hood, the quarantine information is still present -- it's just the GK flags that evolve. (At least on my system and presumably on the systems of other folks at Mozilla who have investigated. Folks would have noticed if the xattr data disappeared entirely, or the events were no longer present in the event database). In principle we can fish the GK GUID from getxattr and fish the quarantine event data from the database, and in practice the commit here achieves this.

Therefore, I think there are a few things to be done:

  1. Test this approach a little more generally, especially on macOS 10.15 Catalina (I'm running 10.14 Mojave), with a try build. Maybe there's something special on my laptop -- lord knows I've monkeyed with SIP often enough.

  2. Decide if we're okay fishing a macOS system database in this way. Most of the places on the web that discuss this are at least 5 years old, so it doesn't seem to be a rapidly evolving area. But it's not exactly supported.

  3. Clean up the approach, possibly by pushing this implementation approach into native code. However, there's no other non-test caller of getReferrerURL, and there's only one non-test caller of setReferrerURL at this time, so my guess is that we could trim this down quite a bit if we chose to by moving more things to JS.

  4. Consider caching the extracted attribution codes on disk outside of the system quarantine DB, potentially in the install directory like we do on WIndows (so that this isn't profile-specific). Folks do purge their quarantine database, although presumably most don't; and querying an SQLite DB isn't cheap, so we'd be more robust and more performant with a little bit of JSON on disk.

Just quick thoughts.

The data doesn't disappear, it was just in accessible via gui.

Maybe a one-time fish to copy the data into a json file for future use, but I wouldn't do it every startup. IIRC this stuff is read early.

If the fishing could happen in an installer all the better.

Time for dinner.

While we're here, push this out to the 85 cycle, since there's no
reason to revisit this for the 80 cycle.

The next step is to have the cached attribution file location vary
depending on the platform, and to have the tests run on additional
platforms.

Depends on D84263

A malformed URL returns null. The subsequent call to
CocoaFileUtils::AddQuarantineMetadataToFile does handle null
correctly, so this just avoids a crash.

Assignee: nobody → nalexander
Status: NEW → ASSIGNED

The existing code handles Windows and macOS quite differently. On
macOS, the tests were not as comprehensive; this patch brings them
level and makes the form of the attribution data uniform on Windows
and macOS. In particular, attribution data fields will now be URI
coded, just as they are on Windows.

This will allow us to re-use the parsing machinery we have on Windows
when we cache attribution codes on macOS.

Depends on D92691

The issue this is addresses is that
CFURLCopyResourcePropertyForKey
does not return quarantine data when launched as a GUI App. What
happens is that launching via the GUI requires the user to override
GateKeeper by going to Security & Privacy > Open Anyway. Doing that
updates the GateKeeper flags, and then the macOS API denies access:
once the GK flags reach some state, quarantine information is not
returned. This is not documented (as far as I can see) but moons ago,
somebody else on the internet witnessed the same
thing
.

To work around, we run the system SQLite binary, to fish the relevant
information out of the per-user quarantine database. (SQLite is
installed by default on all relevant macOS versions.)

The most significant security concern I see is whether we can trust
this binary (in /usr/bin/sqlite3). Some discussion within the
Install/Update team suggested that an attacker who could corrupt or
modify that binary already had write access to the disk, which is an
attack vector equal to a totally compromised Firefox. If we determine
that we can't use the system SQLite binary, then we could use
Firefox's compiled copy of SQLite, but we might see versioning
issues. The system SQLite binary feels more robust.

This is implemented as a JS component for convenience, mostly: there
is no API for capturing output from nsIProcess. It would be
possible to maintain the existing XPCOM contract by renaming the
existing contract and adding a contract with a JS implementation that
passes through to the renamed implementation, but it doesn't seem
worth the effort.

In the next commits, we will generalize the existing caching mechanism
form Windows to also apply to macOS. This is mostly a performance
optimization, so that we sniff a single well-known location rather
than launching a process at each startup, although there is a
correctness argument here as well, since the quarantine database is
dynamic and the attribution URL could expire.

Depends on D92692

Attachment #9164898 - Attachment description: Bug 1525076 - Part 2: Expose `AttributionCode.attributionFile`. r?mossop → Bug 1525076 - Part 2: Expose `AttributionCode.attributionFile`. r?mixedpuppy

On Windows, we parse the attribution data as if it was the
URL.search parameter. On macOS, we parse a URL slightly
differently: we parse it like URL.searchParams and we deduplicate
parameters.

For caching on macOS we need to serialize the parsed data. We could
use either approach but we elect to write the code serialized as it
would be on Windows, since that is the one that is getting actively
tested in the wild.

Depends on D84264

Querying the macOS system quarantine database is relatively slow, so
we'd like to do it only once.

We must make BROWSER_ATTRIBUTION_ERRORS histogram apply to macOS as
well as Windows. We add error codes to capture macOS-specific
detalis. And we push this out to a later cycle, since there's no
reason to revisit this immediately.

Depends on D92694

Attachment #9164897 - Attachment is obsolete: true

mixedpuppy: OK, I think this is ready for initial review. I've left extremely verbose logging in place while I green up try (which should show up, eventually, at https://treeherder.mozilla.org/#/jobs?repo=try&revision=9ccc94fcf5a353d94235eec3d93bdcfca93a8f0e) although all tests are passing locally and many intermediate builds continue to be green on try. I'm happy to take direction on logging: leave it (but drop the displayed level), remove it entirely, make it more succinct.

I'll include detailed instructions for our QA testers on this ticket sometime in the next day or two.

Attachment #9164899 - Attachment is obsolete: true
Attachment #9151618 - Attachment is obsolete: true

Comment on attachment 9180048 [details]
Bug_1525076_data_review.txt

Megan -- you reviewed this initially in Bug 1621402.

Attachment #9180048 - Flags: data-review?(nalexander) → data-review?(mmccorquodale)

(In reply to Nick Alexander :nalexander [he/him] from comment #37)

Comment on attachment 9180048 [details]
Bug_1525076_data_review.txt

Megan -- you reviewed this initially in Bug 1621402.

Relevant change to Histograms.json is in https://phabricator.services.mozilla.com/D92695#change-pNeGXkABjDm2.

To test attribution:

  1. Paste the following into ~/Downloads/setattrs.sh:

    #!/bin/bash
    
    # Sets the quarantine attributes and refererr URL for a download on macOS.
    # setattrs.sh 'https://example.com?content=URI%20encoded&source=also%20encoded' /path/to/App.app
    
    REFERER=$1
    shift
    GUID=`uuidgen`;
    
    # GUID="025EE793-945F-463A-B3D3-F052F62926C8"
    
    # https://eclecticlight.co/2017/12/11/xattr-com-apple-quarantine-the-quarantine-flag/ has different flags:
    # xattr -w com.apple.quarantine "0083;5991b778;Safari.app;BC4DFC58-0D26-460D-9688-81D119298642" appname.app
    for i in "$@" ; do
    # xattr -w com.apple.quarantine "01c1;5c53622a;Chrome;$GUID" "$i"  # Allowed?
    xattr -w com.apple.quarantine "0181;5c53622a;Chrome;$GUID" "$i"  # Quarantined.
    done
    
    sqlite3 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV2 "INSERT INTO \"LSQuarantineEvent\" VALUES('$GUID','570995305.0','org.mozilla.nightly','Nightly','http://dummy.com/file.zip',NULL,NULL,0,NULL,'$REFERER',NULL);"
    
    # To clear:
    # xattr -r -d com.apple.quarantine "$1"
    
  2. Download a fixed DMG. I have chosen the "OS X Cross Compiled opt" task from https://treeherder.mozilla.org/#/jobs?repo=try&revision=b5813c65d7897ec92f0ba714e802804f40bb008d, namely https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OVgZCrmyTNKOTjiH3b5URQ/runs/0/artifacts/public/build/target.dmg

  3. Open the DMG and copy Firefox Nightly somewhere, say ~/Applications.

  4. Verify that your security settings are default (or at least, not unusually permissive). Open "System Preferences", navigate to "Security & Privacy", and ensure that you have "Allow apps downloaded from: App Store" set. (I think that Nightly builds will be from an identified developer

Let's verify that there's no attribution found with the taskcluster download URL.

This is the case where there's a quarantine GUID, and a URL, but the URL has no attribution.

  1. Start Firefox Nightly as a GUI App -- i.e., do not launch it using the command line with ~/Applications/Firefox\ Nightly.app/Content/MacOS/firefox. You can launch it from the command line using open ~/Applications/Firefox\ Nightly.app.

  2. Witness the "unidentified developer" interstitial. Click OK. Return to "System Preferences > Security & Privacy" and "Open Anyway". Select "Open".

  3. Navigate to about:telemetry, search for "attribution". Witness no results. (This checks for attribution data.)

  4. Navigate to about:telemetry > Histograms, search for "attribution". Witness no results. (This checks for attribution related errors.)

  5. Navigate to about:support, find the "Update folder". For me, that is /Users/nalexander/Library/Caches/Mozilla/updates/Users/nalexander/Applications/Firefox Nightly.

  6. Verify that there is a file macAttributionData in the Update folder. It should be empty, 0 bytes.

Let's verify that we can handle a corrupt macAttributionData file.

  1. Shut Firefox Nightly.

  2. Replace the contents of macAttributionData with garbage. It should now be 7 bytes.

  3. Restart Firefox Nightly.

  4. Return to about:telemetry#search=attribution. Witness BROWSER_ATTRIBUTION_ERRORS has an error (it's index 1, but that's not relevant to this particular test).

Let's verify that we can handle valid attribution.

  1. Disable, shut Firefox Nightly, restart and renable Telemetry, then shut Firefox Nightly again. This is to clear the error from before -- I don't know a better way to do this.

  2. Delete macAttributionData.

  3. Run:

    ~/Downloads $ bash setattrs.sh 'http://example.com?utm_source=bar%23&utm_content=baz%3F' ~/Applications/Firefox\ Nightly.app/
    
  4. Restart Firefox Nightly. You may have to click through the "unidentified developer" interstitial because the quarantine URL has changed.

  5. Return to about:telemetry#search=attribution. Witness data like:

    Environment Data
    attribution.source  bar%23
    attribution.content baz%3F 
    

    Witness that the fields are URI-encoded (this agrees with the representation on Windows).

  6. Restart Firefox Nightly.

  7. Return to about:telemetry#search=attribution. Witness the same data, and no BROWSER_ATTRIBUTION_ERRORS.

Let's verify that we can handle invalid attribution.

  1. Disable, shut Firefox Nightly, restart and renable Telemetry, then shut Firefox Nightly again. This is to clear any errors from before, although they shouldn't have happened.

  2. Delete macAttributionData.

  3. Run:

    ~/Downloads $ bash setattrs.sh 'http://example.com?utm_source=&' ~/Applications/Firefox\ Nightly.app/
    
  4. Restart Firefox Nightly. You may have to click through the "unidentified developer" interstitial because the quarantine URL has changed.

  5. Navigate to about:telemetry, search for "attribution". Witness no results.

  6. Navigate to about:telemetry > Histograms, search for "attribution". Witness no results. There are no attribution related errors because there's no difference between a URL without attribution data and a URL with invalid/malformed attribution data.

  7. Verify that there is a file macAttributionData in the Update folder. It should be empty, 0 bytes.

Finally, let's verify that we can handle unexpected situations with the quarantine database.

First, no database (this will cause an internal invocation of sqlite3 to fail).

  1. Disable, shut Firefox Nightly, restart and renable Telemetry, then shut Firefox Nightly again. This is to clear any errors from before, although they shouldn't have happened.

  2. Delete macAttributionData.

  3. Temporarily move the database to the side:

    mv ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV2 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV2.orig
    
  4. Restart Firefox Nightly. You may have to click through the "unidentified developer" interstitial because the quarantine URL has changed.

  5. Navigate to about:telemetry, search for "attribution". Witness no results.

  6. Verify that there is a file macAttributionData in the Update folder. It should be empty, 0 bytes.

  7. Restore the temporarily moved database:

    mv ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV2.orig ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV2
    

Second, unexpected/missing database contents.

There are two ways to do this. One is to change the GUID so there's no data found in the database; the second is to change the data in the database in some way. Changing the GUID is easier, so let's do that.

  1. Disable, shut Firefox Nightly, restart and renable Telemetry, then shut Firefox Nightly again. This is to clear any errors from before, although they shouldn't have happened.

  2. Delete macAttributionData.

  3. Run:

    xattr -w com.apple.quarantine "0181;5c53622a;Chrome;DB3BA9BC-F132-4706-B3E9-8B5743A9FFFF" ~/Applications/Firefox\ Nightly.app/
    
  4. Restart Firefox Nightly. You may have to click through the "unidentified developer" interstitial because the quarantine URL has changed.

  5. Navigate to about:telemetry, search for "attribution". Witness no results.

  6. Return to about:telemetry#search=attribution. Witness BROWSER_ATTRIBUTION_ERRORS has an error (in index 3).

  7. Verify that there is a file macAttributionData in the Update folder. It should be empty, 0 bytes.

All done!

Attachment #9180041 - Attachment description: Bug 1525076 - Part 2: Add method to serialize attribution data. r?mixedpuppy → Bug 1525076 - Part 3: Add method to serialize attribution data. r?mixedpuppy
Attachment #9180042 - Attachment description: Bug 1525076 - Part 3: Cache attribution code on macOS. r?mixedpuppy → Bug 1525076 - Part 4: Cache attribution code on macOS. r?mixedpuppy

Comment on attachment 9180048 [details]
Bug_1525076_data_review.txt

  1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?
    Yes this will be documented in the probe dictionary.

  2. Is there a control mechanism that allows the user to turn the data collection on and off?
    Yes, users can opt out of telemetry collection.

  3. If the request is for permanent data collection, is there someone who will monitor the data over time?
    Not permanent data collection.

  4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
    Category 1, technical data.

  5. Is the data collection request for default-on or default-off?
    Default on.

  6. Does the instrumentation include the addition of any new identifiers?
    No new identifiers.

  7. Is the data collection covered by the existing Firefox privacy notice?
    Yes.

  8. Does there need to be a check-in in the future to determine whether to renew the data?
    Check in at Firefox 90.

  9. Does the data collection use a third-party collection tool?
    No.


data-review +

Attachment #9180048 - Flags: data-review?(mmccorquodale) → data-review+
Attachment #9164898 - Attachment description: Bug 1525076 - Part 2: Expose `AttributionCode.attributionFile`. r?mixedpuppy → Bug 1525076 - Part 2: Add `AttributionCode.writeAttributionFile`. r?mixedpuppy
Pushed by nalexander@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/e6b09ef2a26c Pre: Avoid a crash in `CFRelease`. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/c1629391c635 Part 0: Test more scenarios on macOS; handle URI component encoding. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/d0c81fd932c7 Part 1: Fix macOS native `getReferrerUrl` by invoking system sqlite binary (from JS). r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/e7e05723ceae Part 2: Add `AttributionCode.writeAttributionFile`. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/4f2c3fe9ee78 Part 3: Add method to serialize attribution data. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/25fd5f2e276e Part 4: Cache attribution code on macOS. r=mixedpuppy
Backout by csabou@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/2337bf23a953 Backed out 6 changesets for OSX xpcshell failures on test_ASRouterTargeting_attribution. CLOSED TREE

Oops, I missed a test. I'll address that. But the test itself is busted: it doesn't await an async function, and therefore races -- and now we do actual asynchronous work so that we always lose the race.

Flags: needinfo?(nalexander)
Pushed by nalexander@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/95fd52531439 Pre: Avoid a crash in `CFRelease`. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/f7d26b270884 Part 0: Test more scenarios on macOS; handle URI component encoding. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/963c8d33d779 Part 1: Fix macOS native `getReferrerUrl` by invoking system sqlite binary (from JS). r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/5dd08176812c Part 2: Add `AttributionCode.writeAttributionFile`. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/690d730341c6 Part 3: Add method to serialize attribution data. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/912fcc3cb274 Part 4: Cache attribution code on macOS. r=mixedpuppy

A follow-up to comment #c39. Once this has merged to Nightly, the regular download page at https://www.mozilla.org/en-CA/firefox/channel/desktop accepts arbitrary query parameters, and therefore can be used to test attribution. The download links are to https://download.mozilla.org/?product=firefox-nightly-latest-l10n-ssl&os=osx&lang=...

So, to test without attribution, navigate to https://www.mozilla.org/en-CA/firefox/channel/desktop, download Nightly for macOS, install into ~/Applications, and check attribution in about:telemetry.

To test with attribution, navigate to https://www.mozilla.org/en-CA/firefox/channel/desktop/?content=content%20content, download Nightly for macOS, and install into ~/Applications. You should find attribution in about:telemetry.

Pushed by nalexander@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/6c2c2e07b95f Pre: Avoid a crash in `CFRelease`. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/7bde7f6ccbd8 Part 0: Test more scenarios on macOS; handle URI component encoding. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/9178ebc95da2 Part 1: Fix macOS native `getReferrerUrl` by invoking system sqlite binary (from JS). r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/4313cfb9dfcb Part 2: Add `AttributionCode.writeAttributionFile`. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/5163df5430c1 Part 3: Add method to serialize attribution data. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/a425e504ae7e Part 4: Cache attribution code on macOS. r=mixedpuppy
Pushed by nalexander@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/d40c867e7f16 Pre: Avoid a crash in `CFRelease`. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/63c378cca774 Part 0: Test more scenarios on macOS; handle URI component encoding. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/e6d811ad83c4 Part 1: Fix macOS native `getReferrerUrl` by invoking system sqlite binary (from JS). r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/98ffcf3788b7 Part 2: Add `AttributionCode.writeAttributionFile`. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/598a997d23c1 Part 3: Add method to serialize attribution data. r=mixedpuppy https://hg.mozilla.org/integration/autoland/rev/bc1ed215773e Part 4: Cache attribution code on macOS. r=mixedpuppy

Clearing NI since I relanded this -- after a very bumpy landing. Sorry, sheriffs!

Flags: needinfo?(nalexander)

Hello,

Verified the fix using the latest Nightly (84.0a1/20201105045247) and Beta (83.0b8/20201103183834) under macOS 10.15.

Telemetry shows attribution parameters are set and checking the macAttributionData file reveals data along the lines source%3Daddons.mozilla.org%26medium%3Dreferral%26campaign%3Dnon-fx-button%26content%3Drta%3AdUJsb2NrMEByYXltb25kaGlsbC5uZXQ as resulting from triggering the RTAMO flow.

Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: