Closed Bug 1237182 Opened 9 years ago Closed 6 years ago

Get rid of buildbot specific stuff in mozharness

Categories

(Release Engineering :: Applications: MozharnessCore, defect)

ARM
Gonk (Firefox OS)
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: edgar, Assigned: catlee)

References

Details

(Whiteboard: [releng:q22018])

Attachments

(16 files, 3 obsolete files)

(deleted), text/x-review-board-request
Callek
: review+
catlee
: checked-in+
Details
(deleted), text/x-phabricator-request
Callek
: review+
catlee
: checked-in+
Details
(deleted), text/x-phabricator-request
Callek
: review+
catlee
: checked-in+
Details
(deleted), text/x-phabricator-request
Callek
: review+
catlee
: checked-in+
Details
(deleted), text/x-phabricator-request
mtabara
: review+
catlee
: checked-in+
Details
(deleted), text/x-phabricator-request
Callek
: review+
catlee
: checked-in+
Details
(deleted), text/x-phabricator-request
Callek
: review+
catlee
: checked-in+
Details
(deleted), text/x-phabricator-request
Callek
: review+
catlee
: checked-in+
Details
(deleted), patch
rwood
: review+
catlee
: checked-in+
Details | Diff | Splinter Review
(deleted), text/x-phabricator-request
tomprince
: review+
sfraser
: review+
Details
(deleted), text/x-phabricator-request
mtabara
: review+
Details
(deleted), text/x-phabricator-request
tomprince
: review+
Details
(deleted), text/x-phabricator-request
tomprince
: review+
Details
(deleted), text/x-phabricator-request
tomprince
: review+
Callek
: review+
Details
(deleted), text/x-phabricator-request
mtabara
: review+
Details
(deleted), text/x-phabricator-request
catlee
: review+
Details
+++ This bug was initially created as a follow-up of Bug #1215084 +++ Quote: It would be nice if we could fix https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/testbase.py#159 to be taskcluster aware. We'll need to get rid of buildbot specific stuff like this in mozharness anyway.
Depends on: 1209180
Changing the wording slightly, as we'll still need to support buildbot too for the time being.
Summary: Get rid of buildbot specific stuff in mozharness → Abstract out buildbot specific stuff from mozharness core
We no longer need to support buildbot, so we can remove buildbot-specific stuff in mozharness.
Summary: Abstract out buildbot specific stuff from mozharness core → Get rid of buildbot specific stuff in mozharness
Whiteboard: [releng:q22018]
Attachment #8974140 - Flags: feedback?(bugspam.Callek)
Comment on attachment 8974140 [details] Bug 1237182: Removing unused buildbot support https://reviewboard.mozilla.org/r/242428/#review248634 Nothing bad stood out to be here. ::: testing/mozharness/mozharness/mozilla/testing/talos.py:207 (Diff revision 1) > - # no opts, check for '--geckoProfile' in try message text directly > - if self.try_message_has_flag('geckoProfile'): > - self.gecko_profile = True > - > # finally, if gecko_profile is set, we add that to the talos options > if self.gecko_profile: I *think* this branch is NOTREACHED now, but I am not certain.
Attachment #8974140 - Flags: review+
Attachment #8974140 - Flags: feedback?(bugspam.Callek) → feedback+
Keywords: leave-open
Pushed by catlee@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/67d2aa892562 Removing unused buildbot support r=Callek
Assignee: nobody → catlee
Attachment #8974776 - Attachment is obsolete: true
Backed out changeset 67d2aa892562 (bug 1237182) for breaking Decision task. CLOSED TREE Log: https://treeherder.mozilla.org/logviewer.html#?job_id=177910441&repo=autoland&lineNumber=1705 File "/builds/worker/checkouts/gecko/taskcluster/taskgraph/util/schema.py", line 31, in validate_schema [task 2018-05-10T17:30:26.402Z] raise Exception('\n'.join(msg) + '\n' + pprint.pformat(obj)) [task 2018-05-10T17:30:26.402Z] Exception: In test 'raptor-firefox-tp6': [task 2018-05-10T17:30:26.402Z] extra keys not allowed @ data['mozharness']['no-read-buildbot-config'] [task 2018-05-10T17:30:26.402Z] {u'allow-software-gl-layers': False, [task 2018-05-10T17:30:26.402Z] u'build-attributes': {u'always_target': False, [task 2018-05-10T17:30:26.403Z] u'build_platform': u'macosx64', [task 2018-05-10T17:30:26.403Z] u'build_type': u'opt', [task 2018-05-10T17:30:26.403Z] u'kind': u'build', [task 2018-05-10T17:30:26.403Z] u'run_on_projects': [u'all'], [task 2018-05-10T17:30:26.403Z] u'shipping_phase': None, [task 2018-05-10T17:30:26.403Z] u'shipping_product': None}, [task 2018-05-10T17:30:26.403Z] u'build-label': u'build-macosx64/opt', [task 2018-05-10T17:30:26.403Z] u'build-platform': 'macosx64/opt', [task 2018-05-10T17:30:26.403Z] u'checkout': False, [task 2018-05-10T17:30:26.403Z] u'chunks': 1, [task 2018-05-10T17:30:26.403Z] 'description': 'Raptor Firefox tp6', [task 2018-05-10T17:30:26.403Z] u'docker-image': {u'in-tree': u'desktop1604-test'}, [task 2018-05-10T17:30:26.403Z] u'e10s': True, [task 2018-05-10T17:30:26.403Z] u'instance-size': u'default', [task 2018-05-10T17:30:26.403Z] u'job-from': 'raptor.yml', [task 2018-05-10T17:30:26.403Z] u'loopback-audio': False, [task 2018-05-10T17:30:26.403Z] u'loopback-video': False, [task 2018-05-10T17:30:26.403Z] 'max-run-time': 1200, [task 2018-05-10T17:30:26.403Z] 'mozharness': {u'chunked': False, [task 2018-05-10T17:30:26.403Z] u'chunking-args': u'this-chunk', [task 2018-05-10T17:30:26.404Z] 'config': {'by-test-platform': {'default': ['raptor/linux_config.py'], [task 2018-05-10T17:30:26.404Z] 'linux64-ccov/opt': ['raptor/linux64_config_taskcluster.py'], [task 2018-05-10T17:30:26.404Z] 'macosx.*': ['raptor/mac_config.py'], [task 2018-05-10T17:30:26.404Z] 'windows.*': ['raptor/windows_config.py'], [task 2018-05-10T17:30:26.404Z] 'windows10-64-ccov/debug': ['raptor/windows_vm_config.py']}}, [task 2018-05-10T17:30:26.404Z] 'extra-options': ['--test=raptor-firefox-tp6'], [task 2018-05-10T17:30:26.404Z] 'no-read-buildbot-config': True, [task 2018-05-10T17:30:26.404Z] u'requires-signed-builds': False, [task 2018-05-10T17:30:26.404Z] 'script': 'raptor_script.py', [task 2018-05-10T17:30:26.404Z] u'set-moz-node-path': True, [task 2018-05-10T17:30:26.404Z] u'tooltool-downloads': False}, [task 2018-05-10T17:30:26.404Z] u'os-groups': [], [task 2018-05-10T17:30:26.405Z] u'reboot': False, [task 2018-05-10T17:30:26.405Z] 'run-on-projects': {'by-test-platform': {'.*-qr/.*': ['try'], [task 2018-05-10T17:30:26.405Z] 'default': ['try']}}, [task 2018-05-10T17:30:26.405Z] 'suite': 'raptor', [task 2018-05-10T17:30:26.405Z] u'test-name': 'raptor-firefox-tp6', [task 2018-05-10T17:30:26.405Z] u'test-platform': 'macosx64/opt', [task 2018-05-10T17:30:26.405Z] 'tier': {'by-test-platform': {'default': 3, [task 2018-05-10T17:30:26.405Z] 'linux64-ccov/.*': 3, [task 2018-05-10T17:30:26.405Z] 'windows10-64-ccov/.*': 3}}, [task 2018-05-10T17:30:26.405Z] 'treeherder-symbol': 'Rap(tp6)', [task 2018-05-10T17:30:26.405Z] 'try-name': 'raptor-firefox-tp6', [task 2018-05-10T17:30:26.405Z] 'virtualization': {'by-test-platform': {'default': 'hardware', [task 2018-05-10T17:30:26.405Z] 'windows10-64-ccov/.*': 'virtual'}}, [task 2018-05-10T17:30:26.405Z] u'webrender': False} [taskcluster 2018-05-10 17:30:26.787Z] === Task Finished === Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=67d2aa892562cb25bff4a7707b016f20560873a3&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-classifiedState=unclassified&selectedJob=177910441 Backout: https://hg.mozilla.org/integration/autoland/rev/9b3ba34a7a1a77c3d9487c557d13e8691879f877
Flags: needinfo?(catlee)
Pushed by catlee@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/ba91fec47e90 Removing unused buildbot support r=Callek
Pushed by acelists@atlas.sk: https://hg.mozilla.org/comm-central/rev/d288b2be548c Port bug 1237182 - remove buildbot specific stuff. rs=bustage-fix
Backed out for breaking the Windows 2012 Nightlies. "IOError: Can't find buildprops.json in ['.', 'Z:\\task_1526126483\\build\\src\\testing\\mozharness\\configs']!" Backout link: https://hg.mozilla.org/mozilla-central/rev/df5dae2daa1b920a3c94e2647a7bd8c2b4a3d48e Log link: https://treeherder.mozilla.org/logviewer.html#?job_id=178176514&repo=mozilla-central&lineNumber=576
C-C backout: https://hg.mozilla.org/comm-central/rev/2095ab9585866cef1902d32ddcc9137faff6de51 Backed out changeset d288b2be548c: Port of bug 1237182 which was backed out from M-C. a=jorgk
Comment on attachment 8974140 [details] Bug 1237182: Removing unused buildbot support https://reviewboard.mozilla.org/r/242428/#review250182 catlee asked me to look at the interdiff (4-5) and it looks good to me.
Attachment #8974140 - Flags: review+
Pushed by catlee@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/2f6fc4ca009e Removing unused buildbot support r=Callek
Pushed by mozilla@jorgk.com: https://hg.mozilla.org/comm-central/rev/2b3ad9ae9b0d Port bug 1237182 - Remove unused buildbot support. rs=bustage-fix
Flags: needinfo?(catlee)
MozReview-Commit-ID: GN7lGzWwcvx
Attached file Bug 1237182: Remove unused mapper mixin (obsolete) (deleted) —
MozReview-Commit-ID: GN7lGzWwcvx
Do we also need to remove the references to read_buildbot_config ? <https://searchfox.org/mozilla-central/search?q=read_buildbot_config>
Flags: needinfo?(catlee)
(In reply to Bob Clary [:bc:] from comment #23) > Do we also need to remove the references to read_buildbot_config ? > <https://searchfox.org/mozilla-central/search?q=read_buildbot_config> Yes, I've got other patches in the works that do that.
Flags: needinfo?(catlee)
Attachment #8976666 - Attachment is obsolete: true
Comment on attachment 8976663 [details] Bug 1237182: Remove unused mapper mixin Justin Wood (:Callek) has approved the revision. https://phabricator.services.mozilla.com/D1305
Attachment #8976663 - Flags: review+
Attachment #8974140 - Flags: checked-in+
Attachment #8976663 - Flags: checked-in+
Comment on attachment 8977056 [details] Bug 1237182: Get rid of taskcluster helpers Justin Wood (:Callek) has approved the revision. https://phabricator.services.mozilla.com/D1313
Attachment #8977056 - Flags: review+
Comment on attachment 8977057 [details] Bug 1237182: remove mock(chroot) support Justin Wood (:Callek) has approved the revision. https://phabricator.services.mozilla.com/D1314
Attachment #8977057 - Flags: review+
Attachment #8977057 - Flags: checked-in+
Attachment #8977056 - Flags: checked-in+
Comment on attachment 8977058 [details] Bug 1237182: removing some blobber references Mihai Tabara [:mtabara]⌚️GMT has approved the revision. https://phabricator.services.mozilla.com/D1315
Attachment #8977058 - Flags: review+
Pushed by catlee@mozilla.com: https://hg.mozilla.org/mozilla-central/rev/3fc29973e494 Fix android l10n nightlies a=bustage
Attachment #8977076 - Attachment description: Bug 1237182: remove purge mixin r=Callek → Bug 1237182: remove mozharness purge mixin r=Callek
Attachment #8979668 - Attachment is obsolete: true
Comment on attachment 8977076 [details] Bug 1237182: remove mozharness purge mixin r=Callek Justin Wood (:Callek) has approved the revision. https://phabricator.services.mozilla.com/D1318
Attachment #8977076 - Flags: review+
Attachment #8977058 - Flags: checked-in+
Attachment #8977076 - Flags: checked-in+
Comment on attachment 8980077 [details] Bug 1237182: Remove more buildbot references from mozharness r=Callek Justin Wood (:Callek) has approved the revision. https://phabricator.services.mozilla.com/D1372
Attachment #8980077 - Flags: review+
Comment on attachment 8980076 [details] Bug 1237182: Remove BuildbotMixin in favour of AutomationMixin r=Callek Justin Wood (:Callek) has approved the revision. https://phabricator.services.mozilla.com/D1371
Attachment #8980076 - Flags: review+
Attached patch raptor.diff (deleted) — Splinter Review
Attachment #8980642 - Flags: review?(rwood)
Comment on attachment 8980642 [details] [diff] [review] raptor.diff Review of attachment 8980642 [details] [diff] [review]: ----------------------------------------------------------------- LGTM and I see your raptor try push is green, thank you!
Attachment #8980642 - Flags: review?(rwood) → review+
Pushed by catlee@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/5563ed803d7f Remove more buildbot references from mozharness r=Callek https://hg.mozilla.org/integration/mozilla-inbound/rev/c53ad5b24efd Remove BuildbotMixin in favour of AutomationMixin r=Callek https://hg.mozilla.org/integration/mozilla-inbound/rev/d52176ab1ec9 remove buildbot_config from raptor mixin r=rwood
Pushed by catlee@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/9bfce79fc419 Fix python unitests r=bustage on a CLOSED TREE
Comment on attachment 8981458 [details] Bug 1237182: Remove set_property(..., write_to_file) support r=tomprince,Callek Tom Prince [:tomprince] has approved the revision. https://phabricator.services.mozilla.com/D1447
Attachment #8981458 - Flags: review+
Attachment #8980076 - Flags: checked-in+
Attachment #8980077 - Flags: checked-in+
Attachment #8980642 - Flags: checked-in+
Comment on attachment 8981455 [details] Bug 1237182: get rid of oauth.txt Mihai Tabara [:mtabara]⌚️GMT has approved the revision. https://phabricator.services.mozilla.com/D1444
Attachment #8981455 - Flags: review+
Comment on attachment 8981456 [details] Bug 1237182: Clean up codecoverage property helpers Tom Prince [:tomprince] has approved the revision. https://phabricator.services.mozilla.com/D1445
Attachment #8981456 - Flags: review+
Comment on attachment 8982202 [details] Bug 1237182: Remove default_blob_upload_servers r=mtabara Mihai Tabara [:mtabara]⌚️GMT has approved the revision. https://phabricator.services.mozilla.com/D1474
Attachment #8982202 - Flags: review+
Comment on attachment 8982372 [details] Bug 1237182: [mozharness] Remove some obsolete solaris related configuration; r?catlee Chris AtLee [:catlee] has approved the revision. https://phabricator.services.mozilla.com/D1482
Attachment #8982372 - Flags: review+
Pushed by mozilla@hocat.ca: https://hg.mozilla.org/integration/autoland/rev/b36a59489f4d [mozharness] Remove some obsolete solaris related configuration; r=catlee
Comment on attachment 8981454 [details] Bug 1237182: Get rid of buildprops.json Tom Prince [:tomprince] has approved the revision. https://phabricator.services.mozilla.com/D1443
Attachment #8981454 - Flags: review+
Comment on attachment 8981454 [details] Bug 1237182: Get rid of buildprops.json Simon Fraser [:sfraser] ⌚️GMT has approved the revision. https://phabricator.services.mozilla.com/D1443
Attachment #8981454 - Flags: review+
Comment on attachment 8981458 [details] Bug 1237182: Remove set_property(..., write_to_file) support r=tomprince,Callek Justin Wood (:Callek) has approved the revision. https://phabricator.services.mozilla.com/D1447
Attachment #8981458 - Flags: review+
Comment on attachment 8981457 [details] Bug 1237182: Remove unused try message handling r=tomprince Tom Prince [:tomprince] has approved the revision. https://phabricator.services.mozilla.com/D1446
Attachment #8981457 - Flags: review+
Pushed by catlee@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/dab1fa529286 Get rid of buildprops.json r=tomprince,sfraser https://hg.mozilla.org/integration/mozilla-inbound/rev/de5086b44a78 get rid of oauth.txt r=mtabara https://hg.mozilla.org/integration/mozilla-inbound/rev/76667272ca81 Clean up codecoverage property helpers r=tomprince https://hg.mozilla.org/integration/mozilla-inbound/rev/9061381462ac Remove unused try message handling r=tomprince https://hg.mozilla.org/integration/mozilla-inbound/rev/feda34d6ae78 Remove set_property(..., write_to_file) support r=tomprince,Callek https://hg.mozilla.org/integration/mozilla-inbound/rev/3b9d55ec3055 Remove default_blob_upload_servers r=mtabara
The bulk of the buildbot specific code has been removed from mozharness now. I'm calling this done \o/
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Blocks: 1201085
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: