WHAT YOU'LL LEARN
WHAT YOU'LL NEED
How can you get more people to take your product for a test drive? It's a simple question with hundreds of (not-so-simple) answers.
Meet Seiji Cataldo, Senior Growth Marketing Manager at Trainual. He wanted to increase the percentage of visitors to Trainual’s website who signed up for a trial account . The team had tried several methods to increase signups: video overviews, self-service trials, and live calls with account executives. And while these methods all produced trial signups, there was still a major hurdle to overcome.
The problem to solve: How could Trainual shorten the time to value for prospects during the discovery process? Two things made this particularly challenging.
Trainual is rewriting multiple categories (including employee training, process and policy documentation, and more) into a single easy-to-use solution: business playbook.
Trainual's ICP (ideal customer profile) is "extremely industry agnostic": "We have customers in SaaS and tech, franchise owners, professional services — you name it," said Seiji. All of those people think differently. They use different lingo.
With this in mind, Seiji needed a way to simply and effectively communicate the value that Trainual provides to growing business across a wide variety of industries. There had to be a better way.
Seiji believed in the power of the product’s "Aha! Moment” – that split second in the discovery process when the user sees and understands how Trainual will solve their problem.
He reasoned that shortening the time it takes for prospects to reach that moment would result in higher signup rates and more conversions.
So, what could he do to shorten the time to that “Aha! Moment” during the prospect’s journey? The answer: An interactive clickable demo right on Trainual’s marketing site!
Here's how Seiji explained it: "By interactive demo we mean something that is hands-on. Something self-service. A lightweight PLG motion of our product and what we're able to deliver."
Seiji wanted to get something up and running quickly to validate his hypothesis. He was looking for something to “help to plug this hole and help us get it across the finish line quickly".
They chose Navattic as the engine behind this interactive, PLG-like experience for three reasons:
It was simple to use and implement.
It had the right features.
It provided the support Seiji and his team needed to get this done quickly.
Seiji worked with his CMO, and together, they were able to build the entire demo experience on their own in a matter of days. They included 4-5 paths that best showed how Trainual provides value for growing businesses.
With the interactive demo complete, it was time to test!
Seiji didn't just want the new interactive demo to work. He wanted it to be better than the existing prospect journey, and he wanted the data to back it up.
That's why he set up an A/B split test using Mutiny. With an apples-to-apples comparison, Seiji would have the metrics on hand to understand exactly how much of an impact this new experience would have on trial signups and cohort conversion rates.
The experiment was set up as a 50/50 split on the demo page without channel or campaign segmentation. Just an A/B test across the entire funnel, with the exclusion of mobile devices due to the desktop-oriented UX of the clickthrough demo.
The Control looked like this:
And the variant like this:
Seiji also wanted to see how this experiment would influence down-steam growth metrics, like pipeline, new customers, and product adoption. Working with Trainual’s analytics team, he was able to pass the Mutiny split test data into Segment, allowing him to monitor:
Visitor to trial signup rate
Trial to paid conversion rate
Paid product usage
Navattic code: check. Mutiny integration: check. The only thing left to do was add the <iframe> tag to the demo page and wait for statistical significance.
It didn't take long to see positive results. Within a month, the experiment achieved statistical significance with a 99% confidence interval. And the results? Well, an image is worth a thousand words.
Let's explain this in more detail:
437% lift in the "traffic to trial" conversion rate. This is the number of visitors who sign up for a trial.
That’s a major lift for trial volume, but how did that growth continue as those trials matured?
This is where the "day 7 activated trial lift" (or "activated trials") comes into play. This is the % of trials who hit a "high engagement threshold within the application". In other words, the trials that have taken core actions within the application during their trial period.
For the interactive demo, there was a 106.25% lift in accounts that became "activated" by day 7 of their trial. So the experiment not only produced more trial signups from Trainual’s website traffic, but it also produced significantly more activated trials. At the end of the day, though, what really mattered was whether or not it produced business value. So, did that growth trend from signups and activations continue through the funnel?
The answer was a resounding yes. With a"day 14 converted count" lift of 116.7%, the experiment produced over twice the number of paying customers compared to the legacy experience.
According to Seiji, this experiment effectively halved the cost to acquire those converted customers. At over 100% converted customer lift, Trainual was getting “two customers for one” across all desktop visitors to the demo page.
That's the power of A/B testing.