Before getting started: The stuff covered in this chapter builds on previous chapters (we recommend starting there if you haven't done it already).

🕵️ Your Mission

• Review your running experiences to drive your next round of tests

• Launch an iteration to an underperforming experiment

• Launch an iteration to an underperforming component

• Replicate a winner to a new segment

💡 What you’ll learn

✅ How to develop insights from every experience

✅ What to do with winners, losers and flat tests

✅ How to know when to iterate on a component

✅ How to develop your roadmap based on results

🎒 What you’ll need

✅ Your left brain - we’re gonna get analytical here 🤓

Why iterate?

So far we have focused on launching different types of experiences using different data sources. This week we are going to build the final muscle required for long term success: how to iterate and expand.

The best part about experimentation is you always have a path to success. Every test you run is a learning opportunity rich with customer insights. Even if your test doesn’t reach statistical significance, it can reveal patterns or trends that can inform future tests.

Let’s get to it!

Mutiny’s got your back in identifying iteration opportunities. You’ll see some indications in your dashboard when you have an experience ready for iteration.

Take a look at your dashboard for Mutiny’s recommendations on experiences you can iterate on (here’s how).

Launch 2 experience iterations using these recommendations.

What to do with winners

Got yourself a winner? Not surprised, you badass marketer. Promote the experience to 100% of traffic to maximize your impact.

Now let’s think about how to get even more mileage out of the insight. Was the experience created for a specific vertical that you can replicate to other verticals? Are there other parts of your site you can apply similar personalizations to to expand the impact? This is a great way to quickly stamp out more winners once you find a winning strategy.

Launch a new experiment to test your winning insight on another segment or page.

What to do with losers learners

Experimentation means never losing, only learning 😉

Your natural reaction will be to deactivate the experience. A losing experience is actually a good thing! It means you took a big swing and that you can in fact influence this segment. What's important is that you learn from what didn't work, and iterate until you find something that resonates for this audience.

Mutiny makes iterating on experiences really easy. Simply go back into the editor, revert certain personalizations you think could be causing the change (most likely, your headline or CTA is having the biggest impact).

Try a different angle and click the "Launch" button in the top right of the editor. You will see a modal asking if you want to Keep Data or Reset Results. Click "Reset Results" to track your new changes as a separate iteration.

Your previous iteration results will still be stored on the experience card to reference later.

Pick an underperforming experience and think about what part of your hypothesis was incorrect. Make changes and relaunch it. If you’re stuck, email or slack a screenshot to your team and ask them for ideas. Here are some guidelines on how and when to iterate.

What to do with flat tests

It’s hard to glean any insights from a flat test. So let’s change that.

A few things to think about -

1. Did your experience get enough traffic? If traffic is low, try to expand your segment size by adding in more data sources. If you can’t, try expanding the experience to more pages.

2. Are your changes bold enough? Are the elements you changed above the fold? Take bigger swings to make a splash here.

3. Does the segment convert? It’s totally possible your segment just isn’t converting. For example, maybe you have a lot of student traffic who’s just there to learn or download your free product. Consider changing your goal (like driving more free downloads) or move on to higher impact segments.

What about components?

You’ll want to assess the performance of your components in a slightly different way. Here, we’re looking at clickthrough rate and conversion rate to determine how and when to iterate. Note that your conversion rate depends greatly on the component CTA. A higher commitment CTA like demo request will have a lower conversion rate than a lower commitment CTA, like event registration or asset download.

Here are some benchmark clickthrough rates to give you an idea if things are moving in the right direction:

Banner Good: 1.5%+ Great: 3%+

Side pop Good: 3%+ Great: 5%+

If your clickthrough rate is low, try a more compelling hook or shorter text on your component.

If your clickthrough rate is high but your conversion rate is low, you may have a mismatch with the expectation the component creates and the landing page experience when the visitor clicks through. In this case, you can either change the CTA to go to a different page, or you can try personalizing the landing page to match the component content when users click through.

Launch an iteration on an underperforming component to drive stronger engagement.

Looking strong on both clickthrough and conversion? Replicate the strategy to more pages or segments.

What’s next?

You finished the guide and are ready for graduation! Keep practicing the skills you learned, and follow the data to chase and convert those leads.

You should be checking in on your results at least twice a week to make sure your experiences are performing well, iterating as needed, and replicating wins in new audiences based on your learnings.

If you need any help, we’re standing by! We can’t wait to see how you grow with these new tools in your belt (& brain).