Data, Diversity, and Design

In his best-selling 2005 book Blink: The Power of Thinking Without Thinking, Malcolm Gladwell’s discusses how humans think without thinking. Choices that seem to be made in an instant—in the blink of an eye—actually aren’t as simple as they seem. 

How does this process impact the digital experience? Does diversity in design make a difference? What key role does design play in this process? And if so, how do we measure this and tie it to meeting and exceeding business goals? 

These were some of the questions we tackled earlier this month at the dmi: Diversity in Design conference in Washington D.C. Smart Panda Labs Co-Founder Cheryl Myers and I led a session on how design—in particular, design representative of diversity—can and should be informed by data gleaned from digital experimentation.

Rapid cognition and thin-slicing

We began our session with an anecdote Gladwell presents in his introductory chapter of Blink. In 1983, an art dealer named Gianfranco Becchina approached the J. Paul Getty Museum in California claiming to have a marble statue known as a “kouros,” dating from the sixth century B.C. Becchina’s asking price for the statue was $10 million. The Getty took the kouros on loan and began a thorough investigation to authenticate it. From scientific evidence of its age to the bevy of documentation of the statue’s recent history and provenance, there was ample proof of the statue’s authenticity. The Getty concluded its investigation and agreed to buy the statue.

The kouros went up on display, receiving glowing reviews. However, the statue did not look right to a few people – namely an Italian art historian Federico Zeri (who served on the Getty’s board of trustees), Evelyn Harrison (a foremost expert on Greek sculpture), and Thomas Hoving (the former director of the Metropolitan Museum of Art in New York). They were each taken to the see the sculpture, and in what seemed like an instant, they all came to the conclusion that there was something off about the sculpture. All concluded that it was a fake.

The Getty launched a further investigation and found inconsistencies in the documents that supposedly proved the kouros’ provenance. It discovered that the statue actually most resembled a forged kouros that came from a workshop in Rome in the early 1980s. It turned out that dolomite could be aged in a matter of a few months using potato mold. The sculpture was indeed a fake.

“When [the art historians] looked at the kouros and felt an ‘intuitive repulsion,’ they were absolutely right,” writes Gladwell. “In the first two seconds of looking—in a single glance—they were able to understand more about the essence of the statue than the team at the Getty was able to understand after fourteen months.”

At the heart of Blink is the concept of rapid cognition, or “thin-slicing,” the process by which people make quick assessments using a limited amount of evidence. For better or worse, a staggering number of our decisions result from thin-slicing and instinctive hunches about how to act. While the conscious mind is good at studying a wide range of evidence and drawing conclusions from it, our “adaptive unconscious” is adept at assessing a very small amount of evidence about the external world—a “thin slice”—and then forming an instinctive response.

Gladwell is clear in the fact that rapid cognition is often imperfect and sometimes dangerous. After all, this how many prejudicial decisions are made. However, he argues that rapid cognition plays a valuable role in human behavior—a role that’s too-often ignored.

Designing with diversity in mind

As part of a firm specializing in optimizing digital experiences, my colleagues and I must be keenly aware of the rapid cognition and thin slicing that happens as a very natural part of digital engagement. Just as the art and antiquities experts brought their own expertise and personal experiences to bear in their snap judgment of the kouros, consumers are similarly informed by their own knowledge and experience when they interact with a brand’s website, for example. Everything about us, including our ethnicity, gender, geography, and age affect our world view. In our digital exchanges, we must be aware that the impressions made on users may not be the effect intended.

So how does this understanding of human cognition square with our roles as designers and digital strategists? And what do brands and businesses need to bear in mind? Just as our workforces need to be diverse and inclusive in order to better reflect the perspectives of our audiences and consumers, so should our digital experiences reflect the realities of those for whom we are designing.

During our session at the dmi conference, we shared a series of stock photos and website landing pages and asked our audience to share their impressions. The exercise helped to embellish upon our previous discussion on thin-slicing, and it also demonstrated the fact that diversity is relative.

What is diverse to someone from a rural and perhaps less racial diverse area of the country or the world is markedly different from someone from an urban center teeming with diversity. How do you balance such relativity with a desire to make design as personal as possible?

In pursuit of digital experiences that resonate, be data driven

What we see matters. But the question is, how much? Instead of making assumptions about your users, think of yourself as a student of the digital experiences you provide.

Experimentation, testing, and choosing a “learn-it-all” mindset over a “know-it-all” one (see Stanford psychologist Carol Dweck’s best-selling book, Mindset) is winning at some of the largest and most successful companies.

Take Microsoft CEO Satya Nadella, who recently said about the mindset he is implementing at Microsoft: “Some people can call it rapid experimentation, but more importantly, we call it ‘hypothesis testing.’ Instead of saying ‘I have an idea,’ what if you said ‘I have a new hypothesis, let’s go test it, see if it’s valid, ask how quickly can we validate it.’ And if it’s not valid, move on to the next one.”

Or Amazon founder Jeff Bezos, who says, “Our success is a function of how many experiments we do per year, per month, per week, per day.”

Or Mark Zuckerberg, who said of Facebook, “One of the things I’m most proud of, and I think the key to our success, is this testing framework we’ve built.”

If you want to understand to what degree diversity plays a role in the products or services you’re offering, test it, and let the data reveal the answer. For example, change the images on your site to demonstrate differing kinds of diversity, such as gender, ethnicity, age, ability, and intersectionality—overlapping aspects of social categorizations—as much as possible. You may also want to highlight ADA compliance, as another example. Facebook data may be helpful to you in terms of understanding some of the interests and perspectives of your target audiences, and you can consider including some of that content on your site. Throughout this process, we recommend keeping your key performance indicators (KPIs) top of mind and maintaining authenticity—your goal here is to surface diversity without being disingenuous.

Now it’s time to put your efforts to the test. Here are the five steps we suggest in the experimentation process:

1. Define your audiences
2. Consider what diversity is for each audience
3. Test—A/B testing, focus groups, and usability labs are all examples of types of test
4. Read reactions, not explanations (think “adaptive unconscious” vs. conscious)

On this latter step, the point I am trying to make is that a user’s initial reaction, in the form of a rating, for example, is more useful data respective to a digital experience than a conscious explanation; that instant reaction more closely mirrors how decisions are made in such a context. In Blink, Gladwell shares examples of how this works in other contexts as well.

The impact of the changes you are testing can be measured in many ways, such as overall satisfaction (feedback, surveys, net promoter scores), site engagement, social media engagement, and conversion rates. Analyze the data to see if changes you’re making to your digital experience are moving the needle and helping you meet your KPIs.

Then, use your findings to evangelize the value of diversity throughout your organization.

KEY TAKEAWAYS

• Rapid cognition plays a valuable role in human behavior and has a lot to do with how consumers experience digital. “Thin slicing” happens as a very natural part of digital engagement.

• Everything about us, including our ethnicity, gender, geography, and age affect our world view. In our digital exchanges, we must be aware that the impressions made on users may not be the effect intended.

• If you want to understand to what degree diversity plays a role in the products or services you’re offering, test it, and let the data reveal the answer.

• The impact of the changes you are testing can be measured in many ways, such as overall satisfaction (feedback, surveys, net promoter scores), site engagement, social media engagement, and conversion rates. Analyze the data to see if changes you’re making to your digital experience are moving the needle and helping you meet your KPIs.

• Just as our workforces need to be diverse and inclusive in order to better reflect the perspectives of our audiences and consumers, so should our digital experiences reflect the realities of those for whom we are designing.

• Use your findings to evangelize the value of diversity throughout your organization.