Closed
Bug 1173430
Opened 9 years ago
Closed 9 years ago
analyze success of Thank You page project phase 1
Categories
(Input Graveyard :: Submission, defect, P2)
Input Graveyard
Submission
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: willkg, Assigned: willkg)
References
Details
(Whiteboard: u=user c=feedback p= s=input.2015q2)
On June 10th, 2015, we pushed out and turned on the new Thank You page suggestions which currently have a single suggestions provider which uses SUMO Suggest API to find relevant kb articles for user's feedback.
In bug #1133774, we covered what questions we're using to think about success, what metrics we're gathering and how to manipulate those metrics to determine how things went.
This bug covers doing that analysis work and writing up a report with conclusions and possible next steps.
This bug shouldn't be worked on for a couple of weeks so we can accumulate enough data. Maybe it's best to do at the end of June 2015.
Assignee | ||
Comment 1•9 years ago
|
||
This blocks the Thank You page project phase 1 tracker.
Blocks: 1007840
Assignee | ||
Updated•9 years ago
|
Priority: -- → P2
Assignee | ||
Comment 2•9 years ago
|
||
Grabbing this one to work on this week.
Assignee: nobody → willkg
Status: NEW → ASSIGNED
Assignee | ||
Comment 3•9 years ago
|
||
Checked the Event Flow for sumo_suggest1 events in Google Analytics for the last week.
Total engagement:
605 / 1690 = 35.7% people clicked on something
That's well north of 20%, so I think this is a success and that we are solving a real problem here.
Success of the suggestions heuristics:
530 / 1690 = 31.3% people clicked on a suggested kb link
vs.:
75 / 1690 = 4.4% people clicked on "Ask a question in our support forum"
31.3% vs. 4.4% suggests our suggestion heuristic is decent--definitely not completely terrible. I suspect we can do better than that. Not many people choose a second link to click on, so it seems like of the three suggested kb links only one is attractive.
Given all that, I claim this project is a success. I have some thoughts on next steps and there's the trigger-keyword suggestion provider, too. All that in other bugs.
I wrote up a blog post:
http://bluesock.org/~willkg/blog/mozilla/input_thankyou_phase1.html
Closing this out.
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Updated•8 years ago
|
Product: Input → Input Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•