When evaluating performance questions, it's really useful to have access to the raw test results instead of just the median, for example. Right now the only way to do that is to go through the resultset data, find the jobids, and then download the full log, and extract the line. It'd be great to have a cheaper and faster way to get to that.

Comment on attachment 8819310 [details] Bug 1316692 - Upload talos results as json artifact; https://reviewboard.mozilla.org/r/99134/#review99426 ::: testing/mozharness/mozharness/mozilla/testing/talos.py:417 (Diff revision 1) > except: > self.exception("Error while validating PERFHERDER_DATA") > parser.update_worst_log_and_tbpl_levels(WARNING, TBPL_WARNING) > > + def _artifact_perf_data(self, dest): > + src = os.path.join(self.query_abs_dirs()['abs_work_dir'], 'local.json') does local.json get produced all the time? ::: testing/mozharness/mozharness/mozilla/testing/talos.py:421 (Diff revision 1) > + def _artifact_perf_data(self, dest): > + src = os.path.join(self.query_abs_dirs()['abs_work_dir'], 'local.json') > + try: > + shutil.copyfile(src, dest) > + except: > + self.exception("Error while copyig local.json results file") nit: s/copyig/copying/ also: please print out the src and dest values here in case we need to debug more info. ::: testing/mozharness/mozharness/mozilla/testing/talos.py:422 (Diff revision 1) > + src = os.path.join(self.query_abs_dirs()['abs_work_dir'], 'local.json') > + try: > + shutil.copyfile(src, dest) > + except: > + self.exception("Error while copyig local.json results file") > + parser.update_worst_log_and_tbpl_levels(WARNING, TBPL_WARNING) will we want to turn a job orange for this? I think so as we will find issues here.

Comment on attachment 8819310 [details] Bug 1316692 - Upload talos results as json artifact; https://reviewboard.mozilla.org/r/99134/#review99426 > does local.json get produced all the time? Yes, as long as sps profiling is not on: https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/run_tests.py#172 > nit: s/copyig/copying/ > also: please print out the src and dest values here in case we need to debug more info. ok, good call > will we want to turn a job orange for this? I think so as we will find issues here. ok, thanks

Comment on attachment 8819310 [details] Bug 1316692 - Upload talos results as json artifact; https://reviewboard.mozilla.org/r/99134/#review99444 thanks you

Pushed by rwood@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/bcd5a036290c Upload talos results as json artifact; r=jmaher