If there is something I really like about Salesforce is that they always listen to the community – the Trailblazers. We are an active contributor that helps to improve the capabilities of the Clouds.
For some time now, the community has been asking for improvements in the way A/B/…/n Testing can be performed in Marketing Cloud. The native feature was not really flexible and it was not integrated with Journey Builder but… with one of the last releases, we are delighted to know that there is a new feature that is going to improve the way we are testing emails inside Journey Builder.
Please welcome to the Path Optimizer activity!
Journey Builder and Email Testing Activity… Finally All-In-One
How does this activity work? It is really well explained in the documentation:
The Path Optimizer activity enables you to test up to 10 variations of a journey path to determine which path performs best. You can let Journey Builder pick the winner automatically based on email metrics such as opens, clicks, or unsubscribes. You can also manually choose a winner based on metrics tracked inside or outside Marketing Cloud.
The main difference between Path Optimizer and the traditional A/B Testing functionality is that now we can integrate email or SMS testing activites in our existing journeys, without having to stop them or testing the emails separately in Email Studio to see what is the version with a better performance.
Exemplifying concepts
Let’s see how it works with an example.
Imagine we have a journey where an email has to be sent to an audience of 3.000 customers. We want to test two versions of the same email to analyze which type of content generates more engagement (in our case, the Open Rate is the main KPI).
First of all, which is the % of the audience has to be selected as the control group? You are right, it depends. The industry, the purpose of the email, the number of the audience… In our case, we have decided to select 20% of the audience.
It means that 600 customers randomly selected will be split (based on desired percentage) through the different paths (up to 10). In this case, we will send Email A to 300 customers and Email B to the remaining 300.
In order to define which is the winning option, we have two options:
- Email Engagement: The winning option is automatically triggered, chosen based on the email with the highest Open, Clik or Unsubscribe rate, after the evaluation period (days or hours).
- Manual Selection: The winning option is manually selected by the user at any time once the journey is active, by just clicking on the Path Optimizer activity.
From my point of view, the way Path Optimizer works may be a bit tricky to understand at the beginning and, from the email tracking and reporting point of view, it is difficult to extract a detailed report with the results of each email (especially to differentiate the tracking from the control group from the rest of the audience) but for sure this is something will be improved with the next releases.
In the meantime, we have other workarounds like the Analytics Screen in the Email Activity in the journey, the Data Views, the SendLog Data Extensions or the Email Tracking and Reports tabs.
Would you like to know more about Path Optimizer and Single Send? I am glad to present my colleague Jeet, who has created this amazing video about the Path Optimizer functionality. Check it out!
Consulting the Cloud 