She wondered if changing “Top matches” to “Newly joined members” would garner more engagement and increase click-throughs to individual profiles.
Soulmates saw a staggering 137% increase in click-throughs from the “Newly joined members” variation.
They have been testing with Optimizely since early 2012 to grow online readership.
According to Kerstin, agreeing on clear goals upfront helps remove the debate aspect when her team looks at test results and also influences ideas for future experiments.
“Of the 14 tests we have run so far on Soulmates, seven have been successful, three have been negative, and four have been inconclusive, in that the differences did not generate big enough changes to reach statistical significance.
Launched in 2004, Soulmates monetizes via paid monthly subscriptions that allow members to interact with other members.
As a dating site, Soulmates’ primary goal is to convert visitors into paid subscribers.
The negative and insignificant results helped her plan for future tests by honing in on new ways to increase conversions.
Setting a hypothesis before building each test makes executing and interpreting experiment results much easier.
Additional user research around Kerstin’s “people first” hypothesis revealed that many site visitors craved more information on Soulmates’ homepage – which displayed photos and quirky taglines for several single men and women on the site.
Kerstin pitted the original homepage carousel against one that replaced the quirky taglines with the age and location of each featured user.
Kerstin coined this approach, a “people first” strategy.