Optimizing User Engagement with 'Hottest Deals' Widget:
Atolls project (Pepper)
Improvement process
Introduction
In the dynamic landscape of web platforms, user engagement is pivotal. This case study delves into a series of A/B tests conducted to enhance user interaction with our 'Hottest Deals' feature.
By transitioning from a traditional widget to a more integrated tab, our objective was to increase visibility and clickouts, ultimately boosting overall user satisfaction.
Goals
Future Sidebar Removal:
Work towards a streamlined user interface by planning the removal of the sidebar.
Implement Onboarding to New Features:
Ensure users are effectively introduced to and can easily find new features.
Introduce a 'Hottest Deals' Tab:
Create a dedicated tab that lists and sorts the hottest deals, making it more accessible to users.
Research Methodology:
To achieve these goals, we employed a rigorous A/B testing methodology:
Control and Test Groups: Users were divided into control and test groups to compare the effectiveness of the changes.
Specific Changes: Various modifications were tested, including the removal of the hottest widget, the addition of a hottest tab, explanatory tooltips, and visual cues.
Sample Size and Demographics: The tests involved multiple countries, with sample sizes ranging from 5,000 to 10,000 users per group.
First A/B test
Approach
The hottest widget was removed, and a hottest tab was introduced. Additional features included a tab explanation and a sort date option.
Proposal design
Current version
Preliminary Findings:
Control Group Performance: The control group slightly outperformed the test group, with approximately 6,411 more clicks (-1.11% change).
User Feedback: Many users reported difficulty finding the hottest tab, leading to a significant decrease in clickouts (-9.28%, 19,082 fewer clickouts).
Quantitative and Qualitative Analysis:
Quantitative Analysis:
Clickouts Comparison: The hottest widget did not allow direct clickouts, but the hottest tab showed higher engagement (2,584 clicks from the thread pages vs. 3,899).
Qualitative Analysis:
User Behavior: Users across different regions relied on muscle memory and were displeased with the missing hottest widget.
Banner Blindness: Many users experienced banner blindness, making it difficult to locate the hottest tab.
Second A/B test
Observations
Increased Engagement: The onboarding group showed a substantial increase in clicks, from 3,476 to 11,493 (a 230.52% increase).
CTR Improvement: The click-through rate also saw a significant improvement, indicating better user interaction.
Conclusion
The A/B tests revealed the critical importance of user onboarding and visual cues in feature adoption.
Proper onboarding and visual enhancements led to a significant increase in user engagement. As a result, we decided to retain a simplified version of the hottest widget while planning for the eventual removal of widgets.
Next step
Given the impressive results, we are going to roll out these updates to all users. The changes are expected to lead to higher engagement and overall user satisfaction.
Contact me: shimaghasemi36@gmail.com