Closed Bug 1538793 Opened 6 years ago Closed 6 years ago

Missing enrollment telemetry for a live experiment

Categories

(Data Platform and Tools :: General, enhancement, P1)

enhancement
Points:
1

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jlockhart, Assigned: bugzilla)

References

Details

Attachments

(1 file)

This experiment went live on March 18 2019:

https://experimenter.services.mozilla.com/experiments/firefox-quantum-bar-experiment-1-xul-vs-html/

https://normandy.services.mozilla.com/api/v1/recipe/716/

However in Grafana the enrollment data only shows 0-1 enrollments rather than the normal healthy amount we expect:

https://grafana.telemetry.mozilla.org/d/3QA87kliz/experiment-enrollment?orgId=1&var-experiment_id=pref-flip-study-firefox-quantum-bar-experiment-1-xul-vs-html-1523433&var-branch_id=All&from=now-30d&to=now

I think the data is being lost somewhere in the pipeline, can someone help figure this out?

So far I've determined that the data exists in the Events dataset, and clients are enrolling correctly (and sending us data). Given we don't seem to be facing actual data loss, I'm punting this until next week. My current suspicion is that there was a change in nightly that's causing failures in parsing pings in the streaming jobs, but not in batch mode.

Assignee: nobody → ssuh
Status: NEW → ASSIGNED
Points: --- → 1
Priority: -- → P2
Priority: P2 → P1

Aha! The event moved to the parent process some time this nightly cycle (vs dynamic). Fix incoming -- thanks for filing the bug!

I restarted the datadog job and I see more reasonable data for that study now. Reassigning to :akomar so he can restart the test tube job on Monday to pick up these changes.

Assignee: ssuh → akomarzewski

TestTube job has been redeployed from master.

Assignee: akomarzewski → ssuh

Enrollment counts look normal now -- thanks :akomar!

Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Component: Datasets: Experiments → General
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: