A/B Testing: Experiments to Optimize Digital Customer Experiences

The pandemic has brought us to a key junction between culture, technology and economic complications. With the rapidly evolving digital economy, we were already barreling to this point, but COVID-19 created a need to standardize and scale digital-first service and communications immediately.

With so much changing every day in this pandemic, being flexible has become a necessity. Everything about the way people learn, shop, buy and socialize has changed dramatically the past year. And because of that, everyone from business leaders to educators have had to become especially flexible to accommodate consumer needs and concerns, while keeping up with CDC guidelines and government regulations.

Part of organizational flexibility has been a deeper commitment to test and try new things, particularly when it comes to digital experiences. Businesses and other organizations that have responded in this way have seen firsthand how valuable it is for them and the people they serve. As our friends over at AB Tasty explain well in their latest report on the virtual experience economy:

“We would advise seeing these [new digital] trends as a springboard. Use them as the fuel to ignite your next strategic planning discussion around how to improve the customer experience on your website, product or mobile app. And then? Test it! Improve it! Iterate on it! And crucially: confirm whether the idea represents uplift.

The ability to measure the impact of an optimization idea on revenue, and to make these decisions backed with data, is one of the most valuable assets in marketing and product optimization. It should always be your North Star.”

At Smart Panda Labs, we’ve seen firsthand how invaluable data-driven personalization and experimentation is for responding to COVID-19 disruption.

Flexibility in Action: Using Proactive Personalization to Increase Conversions

During the early months of the COVID-19 pandemic, one of our clients – a large prominent research university in the Northeast – experienced challenges with enrollment in their in-person executive education courses.

As uncertainty and fear spread, people were hesitant to sign up for new courses or plan for anything days, weeks or months in advance. On top of that, people already enrolled in classes were cancelling left and right, creating front and backend challenges for the educators and administrators. Up until that point, the courses were also all in person so the entire classroom experience needed to be reinvented as well.

Once they converted most offerings to virtual, the goal then became to switch from “reactive” to “proactive” interactions. While going virtual helped open up new options, it also decreased perceived urgency to enroll early. These courses were often attended by executives and managers so there was also uncertainty with job security and company cutbacks that impacted budgets, workload and overall availability. Instead of signing up months in advance as they previously had done, people were now enrolling up until the day before the course began.

The last-minute enrollments prevented the university from being able to effectively plan ahead to meet course demand and optimize the virtual classroom experiences for participants. The university realized that they needed to communicate a higher sense of urgency to drive more conversions for early enrollment. We worked with this client to gather behavioral data and execute A/B testing strategies across emails, landing pages, and paid advertisements for prospective participants. They found that by adding things like countdown clocks and other time sensitive elements, the booking window broadened with participants enrolling early to secure their spot before courses filled up.

Planning for the Unknown

There are still a lot of unknowns in our world right now. It may be hard at times, but the more you can remain flexible and lead with data, the better prepared you will be for whatever comes next. In addition to our example above, AB Tasty’s virtual experience eBook is full of ideas, tips and success stories that you can use for inspiration to optimize your own virtual customer experiences.