How to set up a split test and measure the results!

When you create a piece of content with Bunting, you run through a 3 step process, Visitor targeting > Content > Measure Impact.

To set up split testing, all you have to do is change the drop-down under “Run a split test to measure impact?” on the Measure Impact page to Yes and let Bunting do all the hard work for you. Once you have done this click Save Changes and you are ready to start collecting Results when your Campaign is Activated.

Running a split test demonstration
Running a split test within the Bunting platform.

Where can I see the Results?

In the campaign lists within Bunting when a split test has been set up for the campaign you can see the results on the right-hand site similar to the image below.

If the Split test has not yet reached its confidence target within Bunting then it will still display that the split test between variations is still active.

If you click on one of these split test result figures it will take you to the Split Test Results page for that campaign and will display all the available metrics:

  • Converted Visitors
  • Revenue
  • Cart Products per Visitor
  • Visits per Visitor
  • Average Visit Duration
  • Bounce Rate
  • Product Views per Visit
  • Page Views per Visit
Split Test Results Page
Image showing split test results page within Bunting.

You can even select a specific variation to see the results for that variation at a much more granular level. Showing a graph of results as well as a history of the variation’s performance.

There is an advanced option to alter the amount of traffic that is able to view the split tests. The default for this is 50/50, but this can be catered to the specific campaign you are testing.

Advanced Split Test Screen
Advanced Split Test Results Screen

 

In addition to this, the confidence target can be changed. This is the metric that is calculated by Bunting as the probability for that variation to be the most effective. Once this target is reached it will deactivate the other variations.

All of the traffic is then routed to the winning variation from the deactivated variations and the control group.

To ensure there are no false positives Bunting will only automatically end the split test and show this campaign to all visitors when the confidence target is reached with more than 200 conversions.

Note: Google analytics has been turned on for this Campaign to further measure the impact. If you want to read more about Google analytics event tracking and what information Bunting passes to it click here.