Closed
Bug 1216218
Opened 9 years ago
Closed 7 years ago
autophone data on perfherder is tracked by platform, we should consider tracking by device type
Categories
(Testing Graveyard :: Autophone, defect)
Testing Graveyard
Autophone
Tracking
(Not tracked)
RESOLVED
WONTFIX
People
(Reporter: jmaher, Unassigned)
References
Details
Right now in autophone, we submit a json blob with results upon completion to treeherder. This includes meta data about the device and the build, then specifics about the test (including performance numbers).
The problem I have is when we access data from the perfherder interface it is sorted by branch/platform/test- there is no concept of device type (e.g. nexus-7). As we switch from talos -> autophone we only plan to use a single device. What happens 6 months later when we switch device types or add a second device type?
Right now the platform is taken from the overall metadata, so if we were to change this we could:
* report the overall platform as android-4-1-blah.<device>
* report a custom platform in the performance_data and modify treeherder to use that.
thinking about this more, we should find a way to break out device type in all treeherder views (pushlog format, and perfherder). This would match up to the first bullet point above. In doing that, it would work easily with perfherder, but confuse treeherder pushlog view.
To adjust it we could just make the platform that we post something more complete:
https://github.com/mozilla/autophone/blob/master/autophonetreeherder.py#L236
What other options do we have? What makes sense to do?
Comment 1•9 years ago
|
||
I think the ideal solution might be to expand the scope of MachinePlatform to incorporate a "hardware" field (maybe optionally null to cover the cases where we don't care, which is many/most of them)
https://github.com/mozilla/treeherder/blob/master/treeherder/model/models.py#L105
CC'ing various treeherder devs on this. I'll bring this up in the week's treeherder meeting as well.
Reporter | ||
Comment 2•9 years ago
|
||
checking in on this bug, I would like to figure out what we want to do here in the short term so we can make adjustments before collecting large quantities of data.
Comment 3•9 years ago
|
||
I guess this is another manifestation of bug 1060769?
Agree we should come up with a long term plan for this.
Reporter | ||
Comment 4•9 years ago
|
||
yes, this sounds related.
Reporter | ||
Updated•9 years ago
|
Comment 5•7 years ago
|
||
Autophone is going away. Resolving these to wontfix.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
Updated•3 years ago
|
Product: Testing → Testing Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•