Have you ever wondered how to turn experiments into customer delight?

The first rule of skiing is falling is good. Falling means you’re learning. Falling also means you’re trying and pushing your skills to improve. Similar statements can be applied to product management. Just as learning to ski is tricky, so is learning to accept that a majority of product experiments will not go as planned. What’s important, in skiing and in product management, getting up and trying again. As a Product leader (or ski instructor), you are responsible for creating a space where it is safe to experiment, learn, and try again (yes, I’m a fan of psychological safety).

While some individuals create boards in Pinterest of their dreams and aspirations, I save articles (see post #2). The following articles reflect the approach I am working on with my team as we increase the amount of experimentation in product. To learn more about the practices I am applying, register for the March cohort of People-First Product Leadership.

Optimize to be Wrong, not Right

Barry O’Reilly always has posts that make one stop and think differently, such is the power of unlearning. His themes stand the test of time, as seen from this article dating back to 2017. Instead of seeking to always be right, gain strength from identifying and removing weaknesses right at the start of a project. The end result will be better due to the early stage learnings.

The method Barry recommends is called Set-Based Concurrent Engineering (SBCE) or set based design, in which a team considers multiple ideas rather than a single idea. With each experiment, the team builds up knowledge that allows them to narrow down ideas through eliminating those with unsatisfactory outcomes. The earlier you experiment and obtain learning, the lower the cost to the business in time, energy, and resources.

If you only explore one idea (called point based design), there is less chance for learning. Then when new information arises, you have to start all over again - maybe even go back to the start. With set based design, learning is gained earlier and may require stepping back one space vs. going all the way back to the beginning.

Do Things that Don't Scale

What do you think of when someone says “wizard of oz”? Do you think of the movie?

If you’re a product manager who is a fan of experimentation, running a wizard of oz experiment might come to mind.

In this article by Paul Graham, the founder of Y Combinator, shares advice given to would-be founders - which reminded me of product leaders running diverse teams. He recommends using manual (human-powered) efforts - aka running “wizard of oz” tests - early on. Why?

1. At the early stage you have less to lose.

2. You’ll learn directly from the end users.

3. Insight, both good and bad, will inform how you will do things differently when your audience grows and you gradually switch to less manual methods.

Yes, the idea of an experiment going wrong can be scary, especially when one is an early stage startup. Yet, that’s the safest time to fail. You can launch faster, you can iterate on feedback quicker, and you’ll know what to build as you’ll have the “muscle memory” of doing it yourself.

Consider how long your team can explore an idea manually, and then push them to see if they can do it for a bit longer. Eventually bottlenecks will arise - and that’s a good thing as it means an increasing number of customers are using your offering. When that happens, it’s a signal to start automating the bottlenecks.

Don’t listen to the naysayers who bring up established players. They have simply forgotten the rough starts all companies, even the popular ones today, have faced. But do listen to your team. Ensure they don’t get discouraged when things go wrong. Instead remind them to focus on the potential, the savings of catching gotchas early on, and what will happen when an experiment goes right and customers experience delight.

Eigenquestions: The Art of Framing Problems

Ok, we now know things can, and will go, wrong teams practice experimentation. We know it’s best to experiment early and often. We know serving customers manually in our early stage experiments provides signals which inform subsequent automation and enable growth. How do we determine what to experiment on? Hello to framing.

In this article by the founder of Coda, Shishir Mehrotra, he shares how to approach framing. The three techniques he recommends are

• Technique 1: Eigenquestions

  • "Eigenquestion" is a made-up-word from linear algebra. It is the question where, if answered, it likely answers the subsequent questions as well.

• Technique 2: The Power of a Great Diagram

  • Framing is both a right brained and a left brained activity. Using diagrams with a consistent pattern and coloring system helps create shared understanding.

• Technique 3: Framing with your Team

  • Framing is not meant to be done alone as great ideas can come from anywhere. Great framers find ways to incorporate diverse perspectives.

Apply these techniques to help your team work together more effectively. And, to encourage the right environment for your team to approach framing, he wraps up the article with a few tips.

1. Start with a positive mindset. In Japan, it's good luck to find a problems as it's a chance to improve something. This positive context can be highly motivating. It moves us from a defensive and fearful attitude to an optimistic and proactive one.

2. Search for the underlying logic of the problem. Be both a great anthropologist (capture the details and summarize a discussion) and a great detective (what's the real issue here) by asking “why” until you get to the root of an issue.

3. Acknowledge it when you're stuck. If you get stuck on a problem, recognize the way in which you are stuck. Get perspective through simplifying. Peel back the layers until you reach the core of the problem.

Hypothesis-Driven Development (HDD)

When working through a problem you are dealing with uncertainty. To help navigate, you should have explicit, testable ideas and focus on disproving them quickly so you can turn your attention to exploring a potentially “better idea”.

The author labels this approach as HDD and uses the following diagram to define the conditions as pivot (disproven) or persevere (proven). What’s even better, as you progress an idea that appears to have potential, the diagram still applies.

For determing what to test, the author recommends using the ‘double diamond’ framing of ‘right problem’ vs. ‘right solution’ which I also use for People-First Product Leadership. By applying the double diamond you can address three big questions:

1. How do you test that you’ve found the ‘Right Problem’?

2. How do you test that you’ve found demand and have the ‘Right Solution’?

3. How do you test that you’ve designed the ‘Right Solution’?

The only way to find out is, first to go out and talk to customers and find out! Second, you need to ask non-leading questions. The goal being to determine ‘Who is our customer, and what’s important to them?’.

We’ll wrap by making the seemingly impossible, possible. Here’s a way to make meetings fun!

Ask This Question at Your Next Meeting

What if meetings accelerated the progress they now stifle? Rethink your meeting approach by asking the question: What are you stuck on?

• Inviting reflection to step back and think deeply about what they are really trying to achieve, as well as the barriers getting in the way.

• Communicating that feeling challenged is not an experience that needs to be hidden or feared. It’s expected. That perspective helps to foster resilience.

• Introducing an opportunity for colleagues to provide input — allowing teammates to mentor each other, harness one another’s strengths, discover untapped expertise and nurture greater collaboration.

• Motivating growth as asking the question offers leaders a tool for communicating that pushing yourself and finding your limits is desirable, without coming across as preachy or out-of-touch.