Shamir Duverseau

Why Marketing Experimentation is Crucial for Digital Transformation (Examples + Results)

Experimentation in every industry is critical for development. Items we use every day have iterated to where they are now thanks to experimentation: the car, the computer, the smartphone, and more. 

Experimentation in marketing is no different. Simply put, marketing experimentation tests data-led ideas to see whether they affect a variable, such as product usage, user acquisition, trial sign-ups, etc. 

Many marketers are intimidated by or misunderstand experimentation. Add to that challenges like getting boardroom buy-in and securing budget for experimentation, and you can see why it’s a growth technique that struggles to gain adoption within many organizations.

In this article, I’ll illustrate the importance of marketing experimentation as part of your digital transformation.

Table of contents

Why many marketers ignore experimentation (and miss out on opportunities)

Marketers spend a lot of time collecting and interpreting data. A study conducted by PHD and Warc found that 88% of marketers spend most of their time on reporting tasks, like tracking performance and producing digestible audience insights for stakeholders and other departments. 

From measuring and reporting traffic trends and conversion rates to sending surveys and facilitating focus groups, so much of marketing is about gathering data. Marketers then use these findings to blindly change the customer experience, with the hope of increasing conversions, and hitting KPIs. 

Top Tip: Learn how to use customer intelligence to turn data into actionable insights 🐼

With so much dedication to reporting on the data you already have, it’s understandable that spending additional resources on experimentation makes some marketers nervous. 

Before I get into why experimentation is actually a good thing for digital transformation, let’s take a look at why many marketers avoid experimentation. 

  • Funding: It’s no secret—digital transformation needs investment, including budget for experimentation. However, organizations that don’t make time for testing are missing out. Gartner recently found that high-performing organizations are not only fans of a test-and-learn experimentation culture, they prioritize it.
  • Effort and resources: Businesses are generally more comfortable putting marketing efforts toward campaigns with a history of demonstrating a good outcome. Organizations can be hesitant to direct resources to experimentation for fear of not seeing tangible results. However, experimentation works quite quickly, uncovering new profitable tactics and discarding processes that don’t yield desired outcomes. 
  • Boardroom buy-in: Unsurprisingly, a gut feeling won’t be easy to sell in the boardroom. Initial buy-in may be a challenge, but you may succeed in your experimentation initiative with case studies and success stories (or even better – a free proof-of-concept). Then, it’s a case of earning more trust in the testing process through results.
  • Accessibility and organization of the right data: When data lives in disparate systems or teams, experiments can get messy. In order to run an A/B test on your landing page content strategy, you’ll need to coordinate with the advertising team(s) driving traffic to the page to ensure symmetrical messaging for each audience or campaign. For a culture of testing to work within an organization, data must be organized and accessible for all. 
  • Lack of faith in the process: Marketers on your team may hesitate to run experiments due to a simple lack of faith in the process. It will take some getting used to for those unfamiliar or uncomfortable with a “fail fast” mentality. Time and successful results from data-led hypotheses will help.
  • Lack of autonomy: Experimentation is most effective when your team can move quickly. Jumping through bureaucratic hoops and waiting for feedback for every step in a marketing experimentation project can create delays and frustration. 

Top Tip: Convince stakeholders that marketing experimentation can lead to huge growth with a New York Times case study 🐼

Benefits of experimenting for digital transformation

While I understand why marketers sometimes avoid experimentation (see above), the benefits far outweigh the risks. Sometimes the biggest challenge for those new to marketing experimentation is understanding that experimentation is not a replacement for data-driven decisions.

On the contrary, when done correctly, an experiment is entirely data-driven: it begins with methodology and analysis and concludes with supporting or refuting evidence. Experimentation and data work together to drive better business decision-making.

Here’s an example of a company that used strategic experimentation to produce real results. 

Viceroy Hotels and Resorts

When the Smart Panda Labs team started working with Viceroy, we suggested A/B testing. Like many stakeholders, they were skeptical.

Our team focused on a button on the hotel group’s website that acted as a “submit” button and linked to their booking engine. The goal was to increase room searches by determining which copy led to the highest click-through rate. 

There was a clear winner from the first experiment (“Book Your Room”), which we then pitted against a slightly more descriptive alternative: “Check Availability.” Check Availability won out overall, and this change in something as small as button copy increased room reservations by $30,000 per month. 

When you’re trying something new, it’s best to start small, confirm your hypothesis, and then go for bigger fish. After proving that experimentation had a genuine impact on revenue, we ran a larger test based on a hypothesis about a new landing page. The result was a $600,000 increase in incremental revenue per year. 

Top Tip: Read our full Viceroy case study here 🐼

Types of experimentation to consider

There are many benefits to experimenting, which can be seen across an array of marketing activities. Here are some experiments to consider and the benefits that come with them.

Evaluating cognitive biases

While great marketing is data-driven, there’s also an emotional component to consider. Everyone develops cognitive biases based on our experiences with the world—including your customers. Marketers often leverage these biases to see if they deliver the desired results.

For example, maybe you’re building a hypothesis around CTA conversion rates and decide to test “Your favorited item is selling out—only 5 left!” (employing the bandwagon effect and scarcity mindset) against “10% off your favorite” (the belief that you’re getting an exclusive bargain). 

This experiment may lead you to use a more effective CTA and help you better understand your customer. Are they more socially motivated, wanting your product because it’s popular? Or, are they more motivated by a good deal?

Experimentation doesn’t need to be complex. A/B testing projects can result in real revenue wins. Real estate firm Related Companies increased their lead conversion by 26% using a similar experimentation method. 

Different types of content

Speaking of knowing your customer, it may not come as a surprise to hear that people consume information in very different ways. Rapidly experimenting with different content formats and platforms will ensure you don’t miss any opportunities to reach your target audience.

For example, you may be surprised to learn from market research that your competitors are using TikTok. The move here isn’t to hire a full-time TikTok expert and invest half your budget into marketing on this platform. Instead, run very specific tests based on what you learned. 

There are many additional ways to experiment with digital marketing. Hopefully you can see that experimentation can have real financial results. Even when experiments fail to support your hypothesis, you benefit from a valuable takeaway and a clearer direction, as well as saving your company from a revenue loss if the change had simply been rolled out without testing. 

Top Tip: Learn more about how to leverage your data in our complete guide to digital analytics 🐼

The risks of not experimenting

Ignoring experimentation is detrimental to a digital transformation because without actioning data or analytics, marketing teams are just sitting on reports. 

While your tried and true marketing strategies may keep you afloat, it will be hard to compete when other companies are willing to take risks to accelerate their growth. 

Of course, it’s important not to experiment for the sake of experimenting. Let’s carry on with our TikTok example above. 

If, overwhelmingly, the data shows your audience prefers email marketing and they never use TikTok, experimenting on the video platform would likely be a waste of time. You should only experiment when the data can be interpreted to fit multiple hypotheses, which you would then test to determine the direction for greater marketing efforts. 

On the other hand, if you ignored survey data that showed TikTok is a growing platform of interest, you could be missing out on reaching potential customers in the way they want to be reached. 

How to get started with your marketing experiments

Here are a few key actions you can take to begin accelerating growth with experimentation. 

Set objectives

Identify the specific component of your marketing strategy that would benefit from an experimentation project. From there, set goals and objectives. Be as specific as possible in determining needs unique to your business and competitive landscape. 

If you’re a retailer, you may want to optimize the checkout flow of your ecommerce site based on data that tells you where users are dropping off (perhaps survey results point to customers feeling annoyed they have to log in again). Your goal would be to convert more customers by coaxing them past this drop-off stage. Then you would design experiments based on the data (perhaps by creating a guest checkout option or moving log-in requirements to post-purchase).

Create data-driven hypotheses

We know that experiments should be rooted in some type of data. But data comes in many different forms. Sometimes, you’ll have a lot of it in the form of reports, surveys, performance metrics for past marketing campaigns, etc. 

Sometimes, the data can be based on competitor insights. For example, perhaps several of your direct competitors have gated their webinar content. You learn from customer research that many of your best leads have watched your webinars and find value in them. Your hypothesis for this example might be: putting our webinars behind a gate will lead to more leads because this content is in high demand with our prospects. 

Start small

Stakeholders can be reluctant to experiment. Start small and prove the concept is worth pursuing. Consider A/B testing copy instead of a website overhaul. Test different types of email subject lines. Before pivoting from long-form written content to TikTok videos, run a test to see how they perform against each other. 

Take these findings to the boardroom

When you have a clear understanding of your organization, its competitive landscape, and relevant testing opportunities available to you, it’s time to take your findings to the boardroom. 

Remember, at least one person around the table will likely be skeptical when it comes to experimentation. Come prepared with data-backed hypotheses, as well as success stories from competitors to get them excited. Pick a pilot program to get started with so you don’t overwhelm them with requests for budget and resources. 

Top Tip: Learn how to get boardroom buy-in for your digital maturity initiatives 🐼  

Key takeaways

The best marketing strategies leverage a combination of data and thinking outside the box. While it can feel counterintuitive, the data helps you narrow down what you’re getting creative about—should we be on TikTok or LinkedIn? Should we be driving our content topics by SEO or leveraging influential experts? 

Once you have evidence that a specific marketing activity is where you should be experimenting, then it’s time to create your hypothesis, test, and learn. 

Experimenting is an exciting way to test hypotheses and ensure you’re staying relevant to your customer. With thoughtful, actionable hypotheses backed by data, you’ll be amazed by what you may learn through the experimentation process.


Submit this form to receive a monthly, curated digest of our blogs, filled with helpful thought leadership, innovative trends and industry insights, all delivered straight to your inbox.

  • This field is for validation purposes and should be left unchanged.