Closed Bug 1216218 Opened 9 years ago Closed 7 years ago

autophone data on perfherder is tracked by platform, we should consider tracking by device type

Categories

(Testing Graveyard :: Autophone, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: jmaher, Unassigned)

References

Details

Right now in autophone, we submit a json blob with results upon completion to treeherder. This includes meta data about the device and the build, then specifics about the test (including performance numbers). The problem I have is when we access data from the perfherder interface it is sorted by branch/platform/test- there is no concept of device type (e.g. nexus-7). As we switch from talos -> autophone we only plan to use a single device. What happens 6 months later when we switch device types or add a second device type? Right now the platform is taken from the overall metadata, so if we were to change this we could: * report the overall platform as android-4-1-blah.<device> * report a custom platform in the performance_data and modify treeherder to use that. thinking about this more, we should find a way to break out device type in all treeherder views (pushlog format, and perfherder). This would match up to the first bullet point above. In doing that, it would work easily with perfherder, but confuse treeherder pushlog view. To adjust it we could just make the platform that we post something more complete: https://github.com/mozilla/autophone/blob/master/autophonetreeherder.py#L236 What other options do we have? What makes sense to do?
I think the ideal solution might be to expand the scope of MachinePlatform to incorporate a "hardware" field (maybe optionally null to cover the cases where we don't care, which is many/most of them) https://github.com/mozilla/treeherder/blob/master/treeherder/model/models.py#L105 CC'ing various treeherder devs on this. I'll bring this up in the week's treeherder meeting as well.
Blocks: 1170685
checking in on this bug, I would like to figure out what we want to do here in the short term so we can make adjustments before collecting large quantities of data.
I guess this is another manifestation of bug 1060769? Agree we should come up with a long term plan for this.
yes, this sounds related.
Autophone is going away. Resolving these to wontfix.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
Product: Testing → Testing Graveyard
You need to log in before you can comment on or make changes to this bug.