Andrew Halberstadt [:ahal] Assignee
	Description • 10 years ago

See https://bugzilla.mozilla.org/show_bug.cgi?id=890116#c1

Code coverage builds and tests are running on Cedar. The builds are also uploading the .gcno zip file. The last piece is to get the test jobs to upload the .gcda files.

	Andrew Halberstadt [:ahal] Assignee
	Comment 2 • 10 years ago

Comment on attachment 8501794 [details] [diff] [review]
Schedule linux64-cc on Ash

Ugh, sorry.. attached this to the wrong bug.

	Joshua Cranmer [:jcranmer]
	Comment 3 • 10 years ago

This is a mixin I played with on code coverage tests. Since this is my push-to-try stuff, I've not concerned myself with trying to hide it behind a flag, but this seems to give results for Linux64 fully (including the web-platform-tests, which I haven't gotten before).

Ignore the code-coverage-gcno.zip stuff--I was trying to get Android builds to not think it the installer before I gave up and just renamed the file output.

	Andrew Halberstadt [:ahal] Assignee
	Comment 4 • 10 years ago

Heh, I actually had this very similar patch hanging around locally for awhile. I do plan to get this landed, but it keeps getting bumped by my quarterly goals. With luck, I'll have some time to work on this this quarter.

In the meantime I don't want to block other people from working on it if they want, so temporarily unassigning myself.

	Andrew Halberstadt [:ahal] Assignee
	Comment 5 • 9 years ago

Going to pick this up again. Sorry for the long delay.

	Andrew Halberstadt [:ahal] Assignee
	Comment 6 • 9 years ago

/r/7723 - Bug 1059951 - Code coverage tests should set GCOV_PREFIX, package .gcda files and upload them to blobber, r=jlund

Pull down this commit:

hg pull -r 9fda257cf65774e319a14a5bc438a593d157aedf https://reviewboard-hg.mozilla.org/build-mozharness

	Andrew Halberstadt [:ahal] Assignee
	Comment 7 • 9 years ago

Sorry this adds a new mixin.. if you'd like me to implement it another way, feel free to give me an r- and let me know what you'd prefer!

	Jordan Lund (:jlund)
	Comment 8 • 9 years ago

(In reply to Andrew Halberstadt [:ahal] from comment #6)
> Created attachment 8598289 [details]
> MozReview Request: bz://1059951/ahal
> 
> /r/7723 - Bug 1059951 - Code coverage tests should set GCOV_PREFIX, package
> .gcda files and upload them to blobber, r=jlund
> 
> Pull down this commit:
> 
> hg pull -r 9fda257cf65774e319a14a5bc438a593d157aedf
> https://reviewboard-hg.mozilla.org/build-mozharness

sorry ahal, I'm actually on PTO now until May 13th. Could I 302 this to someone else? Possibly someone from ateam or mgerva/catlee in releng

	Jordan Lund (:jlund)
	Comment 9 • 9 years ago

Comment on attachment 8598289 [details]
MozReview Request: bz://1059951/ahal

https://reviewboard.mozilla.org/r/7721/#review6531

apologies, I'm on PTO until May 13th.

	Andrew Halberstadt [:ahal] Assignee
	Comment 10 • 9 years ago

Comment on attachment 8598289 [details]
MozReview Request: bz://1059951/ahal

/r/7723 - Bug 1059951 - Code coverage tests should set GCOV_PREFIX, package .gcda files and upload them to blobber, r=jgriffin

Pull down this commit:

hg pull -r 6425568fbf5397575f4529caeee4058e413361f3 https://reviewboard-hg.mozilla.org/build-mozharness

	Andrew Halberstadt [:ahal] Assignee
	Comment 11 • 9 years ago

jlund: No worries, have a good break!

jgriffin: this adds the ability to generate .gcda files, I'll need to make a buildbot-configs change to pass in --code-coverage for those jobs next.

	Andrew Halberstadt [:ahal] Assignee
	Comment 12 • 9 years ago

For those curious, here is a try run with everything working together:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4b39871a9eba

Grab the 'code-coverage-gcno.zip' file from the build directory. Grab 'code-coverage-gcda.zip' files from the individual test jobs on blobber (click job and look under blobber urls on the bottom left). Some kind of automated tooling will be needed to download and merge the .gcda files (if desired), but I think this should be done by external process.

I know how to fix the red browser-chrome jobs. Not sure how to fix the make check failure, but it doesn't seem to be hurting anything, so I'll ignore it for now.

	Jonathan Griffin (:jgriffin)
	Comment 13 • 9 years ago

Comment on attachment 8598289 [details]
MozReview Request: bz://1059951/ahal

https://reviewboard.mozilla.org/r/7721/#review6571

lgtm

::: mozharness/mozilla/testing/codecoverage.py:50
(Diff revision 2)
> +        for root, dirs, files in os.walk(self.gcov_dir):

I guess this is necessary because the directory structure in self.gcov_dir is not consistent among platforms or jobs?

	Andrew Halberstadt [:ahal] Assignee
	Comment 14 • 9 years ago

I'm not entirely sure how consistent the directory structure is tbh. Whatever generates the .gcda files puts the entire absolute path into self.gcov_dir, e.g:

/tmp/tmp123abc/build/slave/try-l6-0000000000000000/src/dom/Foo.gcda

We just need to make sure that the gcno.zip and the gcda.zip have the same structure relative to one another, so making the zipfiles start at the topsrcdir seems easiest. If the directory depth is always constant, I could do something like:

d = '/'.join(d.split('/')[5:])

But that seems even more fragile than the os.walk approach.

	Jonathan Griffin (:jgriffin)
	Comment 15 • 9 years ago

Could we just look for the first "src" dir?

	Andrew Halberstadt [:ahal] Assignee
	Comment 16 • 9 years ago

Comment on attachment 8598289 [details]
MozReview Request: bz://1059951/ahal

https://hg.mozilla.org/build/mozharness/rev/bf20a548b325

On irc we decided that pretty much any method of finding the topsrcdir would be fragile, so decided to leave things as is for now. Code coverage is still experimental so we can make adjustments later if this method doesn't work out.

	Andrew Halberstadt [:ahal] Assignee
	Comment 17 • 9 years ago

This got merged to production and the in-tree pointer was bumped.