Closed
Bug 1349070
Opened 8 years ago
Closed 7 years ago
Create views for Sync engine error data
Categories
(Data Platform and Tools :: General, enhancement, P1)
Data Platform and Tools
General
Tracking
(Not tracked)
RESOLVED
DUPLICATE
of bug 1381641
People
(Reporter: adavis, Assigned: tcsc)
References
Details
(Whiteboard: [fxsync][Sync Q3 OKR])
The Sync summary view includes engine error data, but extracting it in re:dash requires a `CROSS JOIN UNNEST`, which is impractical as we accumulate sync pings. This bug tracks creating a Spark view in a format that's more convenient to query.
Reporter | ||
Updated•8 years ago
|
Whiteboard: [sync]
Reporter | ||
Updated•8 years ago
|
Whiteboard: [sync] → [sync][fxsync]
Updated•7 years ago
|
Component: Metrics: Pipeline → Datasets: General
Product: Cloud Services → Data Platform and Tools
Assignee | ||
Updated•7 years ago
|
Assignee: kit → tchiovoloni
Comment 2•7 years ago
|
||
An initial notebook with a breakdown of the new view can be found in [1].
1. follow-up bug should be filed for importing initial data into athena/re:dash
2. dataset should be evaluated for usability
If (2) looks good:
3. notebook should be scheduled on ATMO
4. notebook should be ported to python_mozetl [2] with tests asserting dataset properties
5. job should be scheduled on airflow [3] from python_mozetl
In the near future steps 3-5 are done by the creator of the dataset, but the process could use some better documentation.
Has this dataset been evaluated against any queries yet? If it hasn't, I can help get the ball rolling.
[1] https://gist.github.com/thomcc/5c1cb88eabc5897cd0fb32e0d928e02b
[2] https://github.com/mozilla/python_mozetl
[3] https://github.com/mozilla/telemetry-airflow
Flags: needinfo?(adavis)
Reporter | ||
Comment 3•7 years ago
|
||
Anthony, can you import that data into a table for the last 30 or 60 days so that we can double check it first? Better safe than sorry.
If all is good, you could flush it and then backfill and schedule it moving forward.
Flags: needinfo?(adavis)
Comment 4•7 years ago
|
||
Once the data is generated, it should be easy to file a bug to get it into redash to query against (see bug 1347785 for a reference).
Assignee | ||
Updated•7 years ago
|
Updated•7 years ago
|
Reporter | ||
Updated•7 years ago
|
Priority: P2 → P1
Updated•7 years ago
|
Whiteboard: [fxsync] → [fxsync][Sync Q3 OKR]
Assignee | ||
Comment 5•7 years ago
|
||
Unnecessary given that 1381641 ended up working out.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → DUPLICATE
Updated•2 years ago
|
Component: Datasets: General → General
You need to log in
before you can comment on or make changes to this bug.
Description
•