Designing experiments to alleviate behavioral biases in innovation

Experimentation is a key component of any innovation validation track, regardless of industry, goal, and even budgets. The goal of experimentation is to validate or (in)validate assumptions that we have about the task at hand - be it a new concept, a consumer segment, or a business model.

We believe it’s possible to overcome the barriers that prevent validating innovation strategies. By acknowledging the challenge, we’ll be a step closer to understanding what validation actually means.

We define validation as a way to confirm your assumptions regarding the desirability, viability or feasibility in order to de-risk the success of your innovation – in that order.

The innovation development process is made up of different stages. Validation and experimentation as a concept can be applied to any of them. At any stage, you can encounter a critical assumption that you have been making about your customer, user or market that, if proven wrong, is putting your innovation at risk. This is why we validate and experiment – to alleviate behavioral bias and validate (or invalidate) concepts.

Experiments are composed of many different methods and techniques, and which experiment to use is directly tied to what we want to validate.  Regardless of which experiments are tested, there is a set of generic rules that apply to all – from digital experiments to empathy interviews.

Rule #1: it’s always the why, almost never the how

If you’re skimming this article for the most crucial information…this is it. One of the most common mistakes we see at the moment of experimenting is not digging into the why, and becoming stuck at the what. 

The importance of asking why when designing experiments for innovation

Any human behavior has a why attached to it; a why that alleviates a pain, helps us accomplish a gain, or achieve a job to be done. The reason the why is so crucial to understand is because once we understand why people do something, we can design for it –  design for solutions that become a part of people’s everyday behavior and choices.

Let us tell you a story

A small company needed to offer a new on-the-go breakfast solution to their customers. They narrowed down their targeted customers to the ones who picked up their food every morning before going to work. They came up with a product that wasn’t only a tasty fast solution to the problem, but also had added nutritional value compared to traditional breakfast combos. When presented with the new offer, the customers didn’t nudge – they kept on buying the usual milkshakes.

While designing experiments, coming up with a solution that fixes the problem, and expecting it to work seems to be a universal misconception. This company’s new product wasn’t working. Not because of the what, but because of the how. Customers kept on consuming the product that was easier to carry with them, and that they could eat while on the way to work. If researchers would have asked the right questions – designed for the right why, they would have saved making an unwanted item.

The same is true about experiments. Understanding and designing for why is the key to achieving real behavioral impact.

Rule #2: getting to the reality

Your doctor has probably asked if you eat healthy, at least once. Before you know it, you find yourself responding positively, knowing that more than a few beers are part of your weekend routine. When asked essential questions, customers tend to do this as well. It doesn’t mean they lie, but they aim to answer the questions not as they are, but with their best version of themselves in mind. 

Dealing with the aspirational-self

Most times, consumers not only answer research questions from their aspirational-self perspective; but they’re sensitive to the way these are asked too. An example of this is confirmation bias. Think of the times you’ve been asked if you like something and you’ve answered “yes” without giving it a second thought.

Confirmation bias is a way of asking questions that hints the person to answer towards a certain answer. These types of questions are unable to gather authentic insights, and lead to researches that end up being empty in applicable content.

Getting the real-self to answer

Creating experiments is meant to offer consumers different options. When asking closed-ended questions, the assumption is that the person has given previous thought to the topic. This is rarely the case.

Think of how people lead their daily lives. An overload of information which they have to choose from, since the human brain can only handle so much. The same happens with experimentation. If you take an idea, concept, or phrase, and approach users with them, the answers will be limited to what they’re being shown. “Shop carting” questions is a method to recreate a real-life experience for them, so they have a wide range of options to choose from. 

One of our past experiences within the healthcare industry helped us come up with an interesting example that reveals this behavior. While conducting interviews, we noticed users bringing up the type of content they followed – they chose governmental blogs and scientific journals over popular influencer posts.

In theory, it seemed we had the answers we were looking for. But, we decided not to settle and put it to the test. That’s why we developed an interactive Facebook feed with a series of different kinds of posts. We included everything available on the market, ranging from officially recognized institutions to the latest TikTok dance. Compared to the survey results, you would’ve thought they were, in theory, surprising…but were really not. 

We collected information on users scrolling past the official publications they’d mentioned as their primary source of information. At the same time, we were able to see how most of them lingered on the popular-type of content. Our model proved the first questions were answered from the aspirational-self perspective, while the experiment showed that, in reality, they weren’t eating that healthy after all.

Rule #3: don’t fake it till you make it

Minimum viable products or low-fi prototypes are amazing ways to test products before developing them even further. This experimentation technique elevates the experiment by offering a real-world touch to the testing phase. 

True experimentation is not about pretending something’s there, it’s actually developing a product and studying its performance with real customers. This stage of the process is a valuable and indispensable source of information on a product or service to be launched.

Designing experiments for innovation validation

When it comes to designing experiments, particularly social media ones, it’s common that businesses redirect their users to landing pages that might as well have been designed by toddlers with access to Sharpies. Not offering high quality interphases during testing stages leads to users dropping before engaging. This is acceptable from time to time, for example when the experimenting is based on clicks. However, secondary layers in testing phases expand on the reality of the experiment.

The best way to leverage this experimental scenario is by working on the way users interact with it. More real-like elements making up the experiment will lead to more detailed input on intent of buy, customer acquisition cost, rate drops, and survey responsiveness. All this by creating a believable shopping cart experiment. 

Investing in these types of testing environments is especially important when trying to come up with disruptive technologies or products. The rate at which the market is expanding doesn’t always give us enough time to adapt to the newness. Realistic experiments help slow down the idea rampage while making the outcomes more efficient.

Thoughtful experimentation

We’ve established that prototypes will be a crucial aspect that will elevate the quality of the information gathered through experimentation. And for that to be a reality, you should think about incorporating a designer as part of your team. 

A good designer is an invaluable asset when creating an experiment. But it’s inevitable, sometimes we’re working solo. This shouldn’t stop you from experimenting, there are multiple free tools you can use to build a realistic high-end prototype. The time it takes to create these experiments is never wasted, but invested. Just as you can find free resources, you can get creative and realize there are users to test your experiments all around you. Friends, family, experimentation can be done well and on a budget. 

Experiment design and more specifically thoughtful experimentation will help inform your research exponentially.

more on validation and experimentation

The value of experimentation in innovation

A guide to why experimentation and validation is crucial to your innovation concept or business design.

Business and concept validation

Business and concept validation through qualitative and quantitative data – the different methods and how to use them.

Guide: 25 validation experiments

Will your business idea work? Find out. This free guide features 25 validation experiments to help you test your hypothesis & de-risk business decisions.

Case: digital validation sprint with Ferring

Check out this story of how we ran a digital validation sprint for Ferring Pharmaceuticals, to help them prioritize the value proposition of a new digital health solution

Thanks for reading!

I’m Mina Drezner – Experiment and Prototype Lead. Spreading innovation culture is in our DNA – if you liked the read, contribute to our mission by sharing this article.

Like it? Share it!

Share on linkedin
Share on facebook
Share on whatsapp
Share on email
Join 30K+ innovators and receive our latest posts

One step closer to meaningful innovation

Keep yourself in the know, get our Newsletter

Stay updated

Join 30K+ innovators and receive our latest posts

Let's get in touch!

After you fill in this form, one of our experts will reach out and talk to you.