Remove it or invoke it explicitely for Firefox UI tests on releases since Firefox UI tests for nightly jobs will not need it.

Hm, I checked the method in more detail now and stumbled over this comment: "OR it skips it when we use --installer-url --installer-path" That means if we pass in an installer path we don't actually run the rest of the method. Maybe we can indeed leave this method intact? Something which came up lately when I was talking to Rail, we might not have the previous_buildid property in pulse notifications that long anymore due to funsize. So we cannot test updates immediately. I'm not sure yet how this new system would look like and when it will be in place. Maybe Rail can shed more light on us here when he is back.

For context, whimboo wants to move his Firefox UI update tests (which are run for nightly builds) to use mozharness. The determine_testing_configuration() made no sense to him, however, in comment 1 he's wondering if the pulse messages will still work for him. I think if we had a way to query about previous_buildid from somewhere else that would be ideal.

Actually we are almost there. Starting from tomorrow, the nightly builds and L10N repacks won't be having any partial updates. See bug 1173459. Funsize will be generating partials (4 partials going backwards). An example task graph can be found here: https://tools.taskcluster.net/task-graph-inspector/#Z1i-AjOzRAybatqPtW-fpA/ Complete tasks report to the following exchange: Exchange: exchange/taskcluster-queue/v1/task-completed Routing key: #.funsize-balrog.# The second task in the graph (signing) publishes a manifest with some information about the update, see https://s3-us-west-2.amazonaws.com/taskcluster-public-artifacts/wyLyDnMCSiSyMII5nY0FUw/0/public/env/manifest.json The balrog task doesn't publish any manifests, but this can be fixed easily. Also I can add any missing information that you need. If you want to track other statues (failures, etc), see the exchanges listed at http://docs.taskcluster.net/queue/exchanges/ Also, the tasks can be accessed via taskcluster index API (http://docs.taskcluster.net/services/index/) or UI: using hg revision: https://tools.taskcluster.net/index/#funsize.v1.mozilla-central.revision.linux64.d4f3a8a75577.sk.3.balrog/funsize.v1.mozilla-central.revision.linux64.d4f3a8a75577.sk.3.balrog or latest: https://tools.taskcluster.net/index/#funsize.v1.mozilla-central.latest.linux64.sk.3.balrog/funsize.v1.mozilla-central.latest.linux64.sk.3.balrog the same for artifacts: https://tools.taskcluster.net/index/artifacts/#funsize.v1.mozilla-central.revision.linux64.d4f3a8a75577.sk.3.balrog/funsize.v1.mozilla-central.revision.linux64.d4f3a8a75577.sk.3.balrog https://tools.taskcluster.net/index/artifacts/#funsize.v1.mozilla-central.latest.linux64.sk.3.balrog/funsize.v1.mozilla-central.latest.linux64.sk.3.balrog IMHO the easiest way to schedule the tests would be by integrating them into the funsize graph I mentioned above. It'd be much better to test the updates before we publish them. ATM we can "easily" do this for Linux and probably Windows. Mac is not supported by Taskcluster yet. I hope this helps. Feel free to ping me if you want to chat about possible solutions for this.

(In reply to Rail Aliiev [:rail] from comment #3) > Actually we are almost there. Starting from tomorrow, the nightly builds and > L10N repacks won't be having any partial updates. See bug 1173459. It's sad to see that this happened without any warning or information to us. Means from now on we do no longer test any firefox update for Nightly and Dev Edition builds. I hope we can quickly work through that and get it fixed. I assume that we should get this handled in a new bug. > Funsize will be generating partials (4 partials going backwards). An example > task graph can be found here: > https://tools.taskcluster.net/task-graph-inspector/#Z1i-AjOzRAybatqPtW-fpA/ > > Complete tasks report to the following exchange: [..] > IMHO the easiest way to schedule the tests would be by integrating them into > the funsize graph I mentioned above. It'd be much better to test the updates > before we publish them. ATM we can "easily" do this for Linux and probably > Windows. Mac is not supported by Taskcluster yet. > > I hope this helps. Feel free to ping me if you want to chat about possible > solutions for this. If we cannot cover all platforms via taskcluster we will still have to do the updates on our side. I will observe the exchange you pointed out, and file a new bug for any additional work. Armen, I assume that for this use-case we might want to keep the determine-testing-configuration action, so it can be used depending on the type of build. For releases you grab the config and for nightly builds we will return early by checking the installer-path. Would that make sense?

> Armen, I assume that for this use-case we might want to keep the > determine-testing-configuration action, so it can be used depending on the > type of build. For releases you grab the config and for nightly builds we > will return early by checking the installer-path. Would that make sense? Exiting early is fine. I think we should close this bug and simply figure matters on the new one you filed.