(In reply to Romain Testard [:RT] from comment #42)

Agreed we need a solution long term and this is being looked at on bug
1498689 (I need to follow that up with Matt). This will require extra QA on
the stub side and Matt's time to get it done which feels like it won't fit
our timelines on this project though.
For the purpose of this experiment delivering 32 bit to all users regardless
of their OS arch won't make the test invalid from an analysis standpoint and
I assume that eligible users would get migrated to 64 bit at their next
update?

We actually don't have any 32 to 64 bit migration paths in place right now, at least for anyone using newer releases than Firefox 56.0. It's very likely we could add a new migration specific to these funnelcake users, so long as we include the time cost of that when considering alternatives. The QA load was quite considerable prior to the last migration, so that's why I'm looking at ways to avoid it and give users 64 bit if we can.

The custom stub installers I'm suggesting would be much less work than implementing bug 1498689. I'd patch the stub code to change where it downloads the full installer from. Roughly the flow would be

• www.mozilla.org download buttons point to (bouncer products) firefox-stub, firefox-stub-f138, firefox-stub-f139 (via traffic cop, all genuine stub installers, no changes required to bedrock etc)
• each stub detects the OS when it runs and downloads the full installer via bouncer
  • firefox-latest-stub: product=firefox-latest, and os=win64 or os=win (32 bit) (no changes)
  • firefox-stub-f138: product=firefox-latest-f138 & os=win64; or product=firefox-latest & os=win
  • firefox-stub-f139: product=firefox-latest-f139 & os=win64; or product=firefox-latest & os=win

ie, funnelcake stubs install the (existing) funnelcake 64 bit build where appropriate, otherwise the 'vanilla' 32 bit build. We don't need a 32 bit funnelcake this way. The implementation is fairly simple - pushing stub patches into a relbranch to generate signed stub installers (similar to what we did earlier for setup.exe), move the files around, and set up bouncer. This is all en-US only, no change there. We've done something similar to this in the past but QA would absolutely be a good idea.

There is a bit more work in custom stub installers than switching to 32 bit funnelcake, but not that much: 2 days vs 1 if I'm conservative, probably quicker. It doesn't make any difference which we choose if 64.0.2 or 65.0 comes along before the cohort is big enough. Custom stubs with vanilla 32 bit is my recommendation, what do you think ?

agibson, do the bedrock and bouncer details look correct to you ?

Matt, this should implement what I describe in comment #43 for 138 - turn on the funnelcake machinery in the stub, but download the vanilla 32 bit build (which avoids some extra work to create both 32 and 64 full installers with custom setup.exe). I've left the ping alone so we get separate metrics from the custom stubs. Similarly URLManualDownloadAppend because the query arg it adds doesn't seem do anything on www.mozilla.org.

There'd be an almost identical version for funnelcake 139.

If I'm following along correctly here and the funnelcake builds would effectively be custom stub installers which would correctly choose between 32/64bit depending on the OS architecture, then yes this could be a good short term solution. As long as the product details we pass along to bouncer don't need to change (they look the same to me here), then this should be an easy thing for bedrock to accomodate.

If 32-bit is not part of the test, is it important to track metrics ? We'll certainly see a drop in pings from the vanilla stub for 32-bit if 18% of downloads are given funnelcake stubs, but that'll be there for 64 too.

Comment on attachment 9034947 [details] [diff] [review]
[gecko] custom stub patch for 138

Pushed this to a new relbranch (FUNNELCAKE_138_139_STUB_BRANCH) at mozilla-release@62c1a96d83a0.
It should yield a signed stub installer here.

Bumped FunnelcakeVersion to 139 at mozilla-release@bd1c59ce5919. It should yield a signed stub installer here. It's going to need a little more help to produce the stub (due to the way I cancelled unwanted tasks). Will need to rerun these taskIds in sequence: cJQCxEWwRyCZqRyc4oa_yg, Vurr0i7GRD6l1HF5llJ8iA, and F1YuFvswTcKfj4VW-hEIog.

I was testing the custom stub installers and found they worked for 64-bit Windows, but the trailing $ on

  +    InetBgDL::Get "${URLStubDownload32}$" \

caused 32-bit to fail. This is an updated patch, carrying over the f+.

138 pushed at mozilla-release@107c640a2548. TaskIds dO6kiRfMSui-uUi3mDCixQ, ciIxSF5AS-G6Ji7N7w87FA, PH-_ISI-TlWEqyUy2TYU_g, byxtvHRhRgy2KJR3rxw_JA to run.

139 pushed at mozilla-release@97d181973f94. TaskIds PamycqJXTlmK40t-2w0ZdQ, Zqwpyr-HS363aiMKCydWKQ, Qv8eMWBrRbiQiUtzmWSVvA, ZBHr51LXQzauoFSMYKmg-g to run.

I've verified the stub for 138 installs vanilla win32 correctly now. I'll finish up my remaining tasks first thing tomorrow.

The stubs have been moved to

https://archive.mozilla.org/pub/firefox/candidates/64.0.2-candidates/build1/partner-repacks/funnelcake/funnelcake138/v1/win64/en-US/Firefox%20Installer.exe
https://archive.mozilla.org/pub/firefox/candidates/64.0.2-candidates/build1/partner-repacks/funnelcake/funnelcake139/v1/win64/en-US/Firefox%20Installer.exe

and bouncer products firefox-stub-f139 and -f139 to point to those. There are also firefox-latest-f138 and -f139 products to serve the full installer.

I've tested win32 and win64 installs work in VMs, starting from already downloaded stub installers, so I think everything is OK from my point of view. The next step is probably to make sure end-to-end install works from staging www.m.o or demo site, with traffic cop enabled.

Nick and Cosmin, can you please confirm if this requires another round of Softvision QA?

Considering the fact that we have new builds, I think it would be the best to perform another round of testing to make sure that everything works as expected. Since we already have the test cases created, it will take about 2 days to test this.
If everyone thinks that we should perform another round of testing, please send a new PI request for it.

Nick and Matt, you guys can probably best advise if another round of QA is needed. Let me know and I can raise the PI request if needed.

I don't think we would get much out of running another full round of QA for the changes that have been made; nothing in the code for the feature has changed, it's really just about how the builds are packaged/delivered.

question regarding the custom stubinstaller: will these be sending install pings like the regular stubinstaller does? if so, will they be sent to a different location?

They will be sending normal stub pings to the normal location. Those pings will contain the funnelcake ID so they can be identified/filtered.

fantastic!

I agree with Matt - the full installers are known good at this point, we should concentrate on the delivery to make sure we've resolved the issues there.

(In reply to Nick Thomas [:nthomas] (UTC+13) from comment #59)

I agree with Matt - the full installers are known good at this point, we should concentrate on the delivery to make sure we've resolved the issues there.

Thanks. I won't raise a PI request.

I've done some testing (see bug 1493595 comment #46) and the builds and distribution is working properly.

https://github.com/mozilla-partners/funnelcake/pull/76 to cleanup the repack config, now that bug 1493595 comment #64 says we have enough people.