Closed Bug 838657 Opened 12 years ago Closed 10 years ago

Implement a spec reporter for perf runs

Categories

(Firefox OS Graveyard :: Gaia::PerformanceTest, defect, P1)

x86_64
Linux
defect

Tracking

(b2g-v1.3T affected, b2g-v1.4 wontfix, b2g-v2.0 fixed)

RESOLVED FIXED
2.0 S3 (6june)
Tracking Status
b2g-v1.3T --- affected
b2g-v1.4 --- wontfix
b2g-v2.0 --- fixed

People

(Reporter: julienw, Assigned: hub)

References

Details

(Keywords: perf, Whiteboard: [c=automation p=2 s= u=])

Attachments

(1 file, 3 obsolete files)

In Bug 837139 we implemented a JSON reporter that shows a synthetic view of all suites. It's also useful to have a more "human" reporter that shows results as soon as they're run.
Assignee: nobody → felash
Assignee: felash → nobody
Keywords: perf
Whiteboard: [c=instrumentation p=]
I would like to start work in this bug. Wich are the requirements for this implementation? I guess the better ideia is reuse the JSON implementation and show it on reporter. My doubts is: 1. Where this more "human" report will be done? Console/APP or something like that?
Julien, I've runned the test-perf from gaia and I've seen how the current implementation of results are. Could you give some guide where I need to work/study, in terms of code, to create a friendly interface using this the JSON output? In a fast looking, I'll need to work with Marionette, Is it right? Another question is, this JSON output is being stored in another place? I guess that It is being used by Datazilla, and that is the motivation to report as JSON, all right?
You can start taking a look at 'tests/reporters/jsonmozperf.js'. This is the current JSON reporter. And you can create a new reporter based on this. Datazilla would still use REPORTER=JSONMozPerf but tests ran manually could choose the new reporter that you create.
You don't need to work with mationette, all this part is already done. You just need to create a new reporter and specify it when you run the perf test.
(In reply to Anthony Ricaud (:rik) from comment #3) > You can start taking a look at 'tests/reporters/jsonmozperf.js'. This is the > current JSON reporter. And you can create a new reporter based on this. > > Datazilla would still use REPORTER=JSONMozPerf but tests ran manually could > choose the new reporter that you create. Thank you for the direction!
I've done a first proposal in this pull request: https://github.com/mozilla-b2g/gaia/pull/11566 But I have some doubts about this kind of implementation. It has a lot of code cloned from "jsonmozperf.js". 1. What do you think about a reporter that can be more reusable or more generic? 2. Who can I require a review?
1. it should be possible to use inheritance to factorize all this 2. either anthony (:rik) or me
(In reply to Julien Wajsberg [:julienw] from comment #7) > 1. it should be possible to use inheritance to factorize all this So, I'm thinking in create a supertype that stores the success and failures as the super class and 2 especialized class (ConsoleMozPerf and JSONMozPerf) for the report function, because they differ only in this way. There is no problem in change the JSONMozPerf, all right? I'm worry about this.
I have another newbie javascript implementation problem. What is the right way to import a specific script on gaia. I'm having problems with this. I want to make a function "exportable"
julienw or rik, could one of you review my new pull? It's on https://github.com/mozilla-b2g/gaia/pull/11566 I've created a BaseMozPerf that does the base job of a PerfReporter. Waiting for review.
Caio: I added comments on the pull request. Note that you should add a file as attachment, you can put the PR url inside this file, this makes it easier to request reviews.
Attached file pull request link (obsolete) (deleted) —
Julien, I'm with some doubts about implementation. I've commentend them.
Attached file pull request link (obsolete) (deleted) —
Attachment #792535 - Attachment is obsolete: true
Remember to assign this bug for me.
I've modified the code. Ps.: When I push a new code, do I need to say it here in bugzilla?
Assignee: nobody → ticaiolima
(In reply to Caio Lima(:caiolima) from comment #16) > > Ps.: When I push a new code, do I need to say it here in bugzilla? It depends. Usually, when your code is ready to review, you need to flip the flag "review" to "?" on your attachment, and add the bugzilla handle for the reviewer (:julien for me, :rik for Anthony). When we review it, we usually can do 2 different things: * either we remove the "?" or we put a "-", and in that case, you just need to add the "?" again when you're ready; * or we don't change anything and in that case you just need to comment here or on github to say you're ready to get reviewed again.
Comment on attachment 792537 [details] pull request link https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Platform/Gaia/Hacking#Submitting_a_patch has some information about submitting a patch. You don't have to tell every time you push code. Just re-setting the review flag is enough.
Attachment #792537 - Flags: review?(felash)
Comment on attachment 792537 [details] pull request link I'll review this tomorrow if that's ok for you, I'me quite full for today already :/
(In reply to Julien Wajsberg [:julienw] from comment #19) > Comment on attachment 792537 [details] > pull request link > > I'll review this tomorrow if that's ok for you, I'me quite full for today > already :/ No problem for me. :)
julien, ping review
Depends on: 912541
Whiteboard: [c=instrumentation p=] → [c=instrumentation p=][mentor=:julienw]
Status: NEW → ASSIGNED
Whiteboard: [c=instrumentation p=][mentor=:julienw] → [c=automation p=][mentor=:julienw]
Target Milestone: --- → 1.2 C3(Oct25)
Attachment mime type: text/plain → text/x-github-pull-request
Attached file new PR from Caio (obsolete) (deleted) —
We can resume work on this now.
Assignee: ticaiolima → felash
Attachment #792537 - Attachment is obsolete: true
Attachment #792537 - Flags: review?(felash)
Component: Gaia → Gaia::PerformanceTest
Attachment #8350047 - Attachment mime type: text/plain → text/x-github-pull-request
Maybe I should take over this one? I didn't realise this bug existed.
Yep, please do :)
Priority: -- → P3
Whiteboard: [c=automation p=][mentor=:julienw] → [c=automation p= s= u=][mentor=:julienw]
Target Milestone: 1.2 C3(Oct25) → ---
Assignee: felash → hub
Priority: P3 → P2
Whiteboard: [c=automation p= s= u=][mentor=:julienw] → [c=automation p= s= u=]
Whiteboard: [c=automation p= s= u=] → [c=automation p=2 s= u=]
Priority: P2 → P1
Attachment #8350047 - Attachment is obsolete: true
Attachment #8430285 - Flags: review?(felash)
Attachment #8430285 - Flags: review?(eperelman)
Attachment #8430285 - Flags: review?(eperelman) → review+
Comment on attachment 8430285 [details] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/19738 I wonder if we should remove the wrapping [] characters but this looks good enough for me, so r=me with a small nit.
Attachment #8430285 - Flags: review?(felash) → review+
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Target Milestone: --- → 2.0 S3 (6june)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: