Jordan Lund (:jlund) Reporter
	Description • 9 years ago

thanks to progress in https://bugzil.la/1131856, a copy of mozharness is now in the gecko tree and is being used for all buildbot based jobs.

We can now switch over TC to also use the gecko copy.

You have a couple options:

1) if you have a copy of the tree and the rev you are build/testing on the slave:

# just call the script you need from the tree:
`python $gecko_tree/testing/mozharness/scripts/$script # with args` from the same cwd you have been calling mozharness from before. The work_dir (`pwd`/build) can stay the same

2) if you don't have a copy (say maybe for tests):

# you can use Archiver[1] and the client[2] to download a tarball of mozharness based on a specific gecko repo + rev:

`python $BUD/tools_jlund/buildfarm/utils/archiver_client.py mozharness --repo integration/mozilla-inbound --rev 0a1f292ba8cc --destination $CWD/mozharness --debug; python mozharness/scripts/$script # + args`

[1] https://api.pub.build.mozilla.org/docs/usage/archiver/
[2] https://hg.mozilla.org/build/tools/file/default/buildfarm/utils/archiver_client.py

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 1 • 9 years ago

Wander: testing/taskcluster/tasks/build.yml has references to MOZHARNESS_* which I won't remove yet, on the assumption that b2g builds are still depending on them (bug 1187706)

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 2 • 9 years ago

From the patch I'm about to submit (more or less)
  https://tools.taskcluster.net/task-inspector/#24zeN42OSuKwl-ynm4z7dA/0

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 3 • 9 years ago

Bug 1185643: use in-tree mozharness; r?wander r?mrrrgn

This required moving the custom mozharness configs out of the builder image
(which puts them in ~, where mozharness no longer looks) and moves them into
mozharness itself.

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 4 • 9 years ago

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

Bug 1185643: use in-tree mozharness; r?wander r?mrrrgn

This required moving the custom mozharness configs out of the builder image
(which puts them in ~, where mozharness no longer looks) and moves them into
mozharness itself.

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 5 • 9 years ago

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

Bug 1185643: use in-tree mozharness; r?wander r?mrrrgn

This required moving the custom mozharness configs out of the builder image
(which puts them in ~, where mozharness no longer looks) and moves them into
mozharness itself.

	Morgan Phillips [:mrrrgn]
	Comment 6 • 9 years ago

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

The only issue I have with this, is that $WORKSPACE/build/src/testing is hard coded in before $MOZHARNESS_SCRIPT so if someone wants to add a custom path they'll need to add "../../../"

Otherwise, heck yes. :)

	Morgan Phillips [:mrrrgn]
	Comment 7 • 9 years ago

On second thought, we'll probably have more users wanting to just change scripts from within the tree than we will people who want to test their own MH checkout. +1

	Wander Lairson Costa
	Comment 8 • 9 years ago

https://reviewboard.mozilla.org/r/14201/#review12819

lgtm

	Wander Lairson Costa
	Comment 9 • 9 years ago

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

https://reviewboard.mozilla.org/r/14203/#review12823

Ship It!

	Pulsebot
	Comment 10 • 9 years ago

https://hg.mozilla.org/integration/mozilla-inbound/rev/e3c15b9f89b3

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 11 • 9 years ago

I'm game to change $MOZHARNESS_SCRIPT to be relative to ${topsrcdir}, but let's take one step at a time.  Specifically, it'd be great to have gecko, b2g, etc. all using the same approach for that.

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 12 • 9 years ago

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=e3c15b9f89b3&exclusion_profile=false for the sake of verifying I didn't break anything B2G

	Carsten Book [:Tomcat]
	Comment 13 • 9 years ago

seems this broke a lot of stuff like b2g and mulet like https://treeherder.mozilla.org/logviewer.html#?job_id=12197062&repo=mozilla-inbound so backed this out and hopefully this fix the issue and we are able to re-open mozilla-inbound

	Pulsebot
	Comment 14 • 9 years ago

Backout:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2a2f1d5a647b

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 15 • 9 years ago

I'm incidentally adding some debugging to 'mach'; here's some sample output:
  https://s3-us-west-2.amazonaws.com/taskcluster-public-artifacts/7oXTBVAzSyCtTOtO2jgqoA/0/public/logs/live_backing.log

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 16 • 9 years ago

Ah, the failures I was seeing in mach are from bug 1186003, the landing of which caused a permafail for linux builds in TC.  I'm not sure why the manifestation was different than in bug 1187047, but hopefully the fix to both is the same.

	:gerard-majax [PTO 01/08/2023-10/09/2023]
	Comment 18 • 9 years ago

This is blocking me for getting Mulet Gu. When can I hope that this is fixed?

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 20 • 9 years ago

:ted -- we might be able to land this patch more-or-less as-is, even if the desktop builds still don't work (12 days ago, there were 3 or so things causing desktop builds to fail, this among them -- probably more now).  I don't recall why the backout occurred.

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 21 • 9 years ago

Ah, yes -- https://dxr.mozilla.org/mozilla-central/source/testing/docker/tester/bin/entrypoint starts with `test $MOZHARNESS_REPOSITORY`, and due to the mach_commands.py changes in e3c15b9f89b3, that value is empty.

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 22 • 9 years ago

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

Bug 1185643: use in-tree mozharness for builds; r?wander

This required moving the custom mozharness configs out of the builder image
(which puts them in ~, where mozharness no longer looks) and moving them into
mozharness itself.

	Wander Lairson Costa
	Comment 23 • 9 years ago

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

https://reviewboard.mozilla.org/r/14203/#review14995

::: testing/taskcluster/mach_commands.py:288
(Diff revision 2)
>          mozharness = load_mozharness_info()

Can't this be removed?

	Wander Lairson Costa
	Comment 24 • 9 years ago

Comment on attachment 8639464 [details]
MozReview Request: Bug 1185643: use in-tree mozharness for builds; r?wander

https://reviewboard.mozilla.org/r/14203/#review14997

Ship It!

	Dustin J. Mitchell [:dustin] (he/him) Assignee
	Comment 25 • 9 years ago

It can't be removed yet, because testing/taskcluster/tester/bin/entrypoint still refers to MOZHARNESS_RE*

	Pulsebot
	Comment 26 • 9 years ago

https://hg.mozilla.org/integration/mozilla-inbound/rev/be6ff275a904

	Ryan VanderMeulen [:RyanVM]
	Comment 28 • 9 years ago

https://hg.mozilla.org/mozilla-central/rev/be6ff275a904