Open Bug 1043739 Opened 10 years ago Updated 3 years ago

Treeherder should use a structured log as its canonical source of data about test runs

Categories

(Tree Management :: Treeherder, defect, P3)

defect

Tracking

(Not tracked)

People

(Reporter: chmanchester, Unassigned)

References

(Depends on 1 open bug)

Details

As of this week we're uploading raw json logs to blobber for all marionette, marionette webapi, and gaia ui test runs, so that should give us a concrete basis for discussion.

As I understand it, the work in bug 986112 is intended to enable this.
I can't upvote this enough.
There's already a treeherder tracker issue for this: https://www.pivotaltracker.com/s/projects/749519/stories/70575430
The summary, which I forgot to write in the previous comment, is that until we only emit structured messages in mozharness and retire tbpl completely we will need to do something like

BEGIN STRUCTURED LOG id:abcde
[legacy unstructured log]
END STRUCTURED LOG id:abcde blobber:http://structured/log/url

Then treeherder would just ignore all the output between the BEGIN and END and grab the data from the relevant structured url instead.

Using blobber as an intermediary here is probably rather inefficient, so once we no longer require the human readable logs for tbpl compat we can change all the mozharnesses to output the structured data directly.
OS: Linux → All
Priority: -- → P3
Hardware: x86_64 → All
Component: Treeherder → Treeherder: Data Ingestion
Priority: P3 → P4
Depends on: 1160117
Component: Treeherder: Data Ingestion → Treeherder: Log Parsing & Classification
No longer blocks: structured-logging
Depends on: structured-logging
Priority: P4 → P3
Component: Treeherder: Log Parsing & Classification → TreeHerder
You need to log in before you can comment on or make changes to this bug.