You can set up an A/B test to measure and compare the performance of different creative groups. For example, you can allocate a portion of the existing budget to see if the new concept or theme of Creative Group B is more efficient than that of Creative Group A.

A/B tests aren't recommended for comparing the performance of different creative formats such as image and video. Given that ad placement varies by creative format, it is tricky to rule out post hoc the possibility that ad placement rather than creative format led to differences in performance.

Set up a creative A/B test 

You can set up A/B tests for individual campaigns using ad groups for those campaigns. Follow the steps below to set up an A/B test.

1. Select an ad account from the top left drop-down menu.

2. Select an app from the left sidebar.

3. Click a campaign title from the list of campaigns under Campaigns.

4. Scroll down to find A/B Test groups. You can see the list of A/B tests created so far. 

5. Click Create. This takes you to New A/B Test where you can configure settings for the new A/B test.

6. Enter a title for the A/B test. Description is optional. 

7. For Variable, select Creative Group from the drop-down menu.

8. Under Test Groups, select an ad group as the control group. You can also see the list of creative groups in the ad group to be used for the A/B test.

   Important: After assigning a control group, you can't change the creative groups for the control group.

9. Select one or more ad groups and creative groups as test groups. You can use the + Add Test Group button to add up to five test groups. Target is randomly divided among the control and test groups to avoid overlap. Be sure that you have created the ad groups to test in advance of setting up the A/B test. To learn how to create an ad group, see ad group settings.

10. Set a budget, key metric, and schedule for the A/B test.

    Feature	Description
    Test budget	Enter the percentage of budget from the campaign to allocate to the A/B test. You can allocate up to 100% of the campaign budget. Test budget is split evenly among the control and test groups. If a different amount of daily budget is set for each day of the week, the daily A/B test budget is adjusted according to the daily budget allocated for each day of the week.
    Key metric	Select the metric to track for the A/B test. You can choose from CPA, CPI, and CPC. Note that CPA is only available for in-app event optimization campaigns and the events used to measure ad group performance is the same as the campaign's optimization target events.
    Schedule	Specify a start date and an end date for the A/B test. Timezone is the same as the ad account timezone. You can change the end date at anytime. A/B tests must run for at least 24 hours but not over 30 days.

11. Click Create to launch the A/B test. Keep in mind that after the A/B test has been activated, you can't change the creative groups and test budget assigned. A/B tests can't be re-activated after they have been completed and you must set up a new A/B test if you would like to test the same creatives again. You can always extend the end date of the A/B test up to 30 days from the start date to collect more data. 

Creative A/B test status and results 

Under A/B Test groups, you can see the status of A/B tests created so far and check out the results of A/B tests that have been completed. Follow the steps below to see a list of A/B tests created so far and A/B test results.

1. Select an ad account from the top left drop-down menu.

2. Select an app from the left sidebar.

3. Click a campaign title from the list of campaigns under Campaigns.

4. Scroll down to find A/B Test groups. You can see the list of A/B tests created so far.

5. Click next to the A/B test of your choice to view the information for creative groups used in the test.
6. To see the results of an A/B test, click the title of the A/B test of your choice. This takes you to the page where you can find all details about the A/B test as well as its results. Under Test Result, you can find the best performing test group that produced significant results and test groups that outperformed the control group.  

   : Best performing test group with statistically significant results

   : Test group that outperformed the control group

   Moloco also provides suggestions for running the A/B test based on information available at the time. As results are likely to vary across the different key metrics, we recommend running a comprehensive analysis of results based on all the available metrics.

   Suggestion	Description
   We need more data	We recommend running the A/B test for a longer period of time to collect sufficient data for analysis.
   We've found a better performing test group	The best performing test group has been identified and there is no need to run another test. The best performing test group is identified only when the confidence level exceeds 75%.
   There is no significant difference	We have identified one or more test groups that outperformed the control group but the difference in performance isn't statistically significant, as the confidence level is below 75%.

   You can compare the following metrics of control and test groups using the dropdown menu on the top left.
   • Spend
   • Impression
   • Click
   • Install
   • Action
   • CTR
   • IPM
   • CPC
   • CPI
   • CPA