Closed
Bug 1298763
Opened 8 years ago
Closed 8 years ago
Create a derived parquet dataset for the sync ping
Categories
(Firefox :: Sync, defect, P1)
Firefox
Sync
Tracking
()
RESOLVED
FIXED
People
(Reporter: markh, Assigned: markh)
References
(Blocks 1 open bug, )
Details
(Whiteboard: [sync-metrics][measure-integrity])
Attachments
(1 file)
(deleted),
text/x-github-pull-request
|
Details |
The next step on the path to dashboards is to create derived parquet dataset.
Dexter tells me the steps are roughly:
- You have to build a new view for the Sync pings (i.e. as done for longitudinal and main summary). You can find an example for the MainSummary at [1] (the same repository also contains the code for the longitudinal).
- This should live in the https://github.com/mozilla/telemetry-batch-view repo.
- I suggest to use IntelliJ IDEA [2] to write your code, as you can easily import the SBT project from the repo and have it download all the dependencies, along with all its code completion cool features.
[1] - https://github.com/mozilla/telemetry-batch-view/tree/master/src/main/scala/com/mozilla/telemetry/views
[2] - https://www.jetbrains.com/idea/
I'll schedule a time to chat with Dexter soon, but getting this on file and so I don't lose the above steps a second time :)
Comment 1•8 years ago
|
||
Mark, do you have any suggestion or additional information about how to add Sync pings to re:dash?
Flags: needinfo?(mreid)
Updated•8 years ago
|
Priority: -- → P1
Whiteboard: [sync-metrics]
Assignee | ||
Updated•8 years ago
|
Assignee: nobody → markh
Comment 2•8 years ago
|
||
There are 2 other smaller steps.
After developing and testing the Sync View code per your description, we'll need to schedule it to run from Airflow[1]. There are several examples in the "dags" directory.
We also need to import the new dataset into the Hive Metastore used by re:dash. This is a matter of adding a crontab entry on the re:dash server (at least until bug 1269781 lands) and should only take a few minutes.
[1] https://github.com/mozilla/telemetry-airflow
Flags: needinfo?(mreid)
Updated•8 years ago
|
Whiteboard: [sync-metrics] → [sync-metrics][measure-integrity]
Assignee | ||
Updated•8 years ago
|
Assignee | ||
Updated•8 years ago
|
Assignee | ||
Comment 3•8 years ago
|
||
Assignee | ||
Comment 4•8 years ago
|
||
This code was merged in https://github.com/mozilla/telemetry-batch-view/pull/114. Bug 1307317 and bug 1307318 are 2 followups that must be done by the pipeline team to finally get this data in re:dash - so all the work here is done.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•