Broadcast A/B tests

What is A/B testing?

A/B testing is a way to simultaneously test versions of your messages among an audience to identify which message performs better.

Can I test more than 2 versions of a message at once?

Yes! You can add more than 2 versions to your A/B tests. Click the + icon to add another version.

Can I send A/B tests to groups or my whole list?

Yes! Select your audience above the compose window like a normal broadcast. Ascend will split the audience randomly so that one part of the audience receives version A and the other receives version B.

How does Ascend split my audience for an A/B test?

Ascend randomly divides the audience based on your configured audience split, which by default is  50/50. You can split your audience up evenly or unevenly.

What metrics can I track once my A/B test is sent?

You can track and compare metrics like delivery rate, response rate and click rate in A/B reports.

Where can I see my A/B testing results?

Review and compare A/B tests by navigating to the A/B test Report from the Broadcast Report Overview page.

Tip: Want to A/B test your automations? Learn more here.

Pick a winner

Optionally, you can run an A/B test on a smaller subset of your audience and pick a winner to send to the remaining audience members.

Automatic: Ascend will automatically pick your winner based on the winning metric you configured.

Automatic (statistic significance): Ascend will automatically pick the statisticaly significant winner based on the winning metric you configured.

‍Manual: Once your results are in you can review them in reports and send the winner to the remaining audience in one click.

What is “pick a winner” in A/B testing?

Pick a winner lets you run an A/B test on a subset of your audience, then automatically (or manually) send the best-performing version to the remaining audience members. This means you no longer have to create a smaller audience subset, review the report, and then manually broadcast the winning version to the rest of your list. Pick a winner only needs to be configured once during A/B test setup.

How does Ascend determine the winning version?

When you set up pick a winner, you choose a winning metric: click rate, response rate, or call rate. You can call the winner after a testing for a duration of time, or at a specific date and time in the future. You can also choose to pick the winner manually or have Ascend pick it automatically.

What happens if the A/B test results are inconclusive?

If you choose automatic winner selection, you’ll be required to select a fallback version. If the results of the A/B test are inconclusive after the test duration or date, the fallback version will be sent to the remaining audience members. You can also optionally toggle on the fallback to send if no statistical significance is found between variants.

Can I pick the winner manually?

Yes! Select the manual option during A/B test setup. Once the test runs, navigate to the A/B test report and click “Pick as winner” in the comparison table for the winner you'd like to send. You’ll be asked to confirm before sending or scheduling the winning version to be sent to the remaining audience. Manual winners can be sent right away or scheduled for later.

Note: sending the winning version manually from the report respects the original quiet hours settings on the broadcast.
‍

Create an A/B test

1. Go to the Broadcast page and toggle A/B Testing to ON
2. Select your version A flow or draft your simple message
3. Click the version B tab and select your version B flow or draft your simple message for version B
4. If you would like to test additional versions click the + icon
5. Click Review Message 
6. Select your audience split between message versions
7. Click Send‍
8. Once your broadcast kicks off, monitor A/B test results in Reports.

‍

Create an A/B test with pick a winner

Use these steps when you want to test variants on a subset of your audience and send the winning version to everyone else.

1. Go to the Broadcast page and toggle A/B Testing to ON
2. Select your version A flow or draft your simple message
3. Click the version B tab and select your version B flow or draft your simple message for version B
4. If you would like to test additional versions click the + icon
5. Click Review Message
6. Select Run A/B test on subset and pick a winner for remaining audience
7. In the Winner column, set the percentage of your audience reserved for the winning version. Then set the percentage of the audience you'd like to recieve the versions of your A/B test.
8. Choose your winning metric from the dropdown: Click Rate, Response Rate, or Call Rate.
9. Choose when to call the winner:
   
   Duration: Set a testing duration up to 72 hours.
   
   Date: Pick a specific date and time in the future to call the winner.
   
   Note: Ascend will mark approximately when your A/B test will complete. We reccomend that you select when to pick a winner after the A/B test has finished sending. Allow time for members to engage with your broadcast before a winner is picked.
   ‍
10. Choose how the winner is picked:
    
    Manual: You’ll pick the winner from the A/B test report once the test duration is up.
    
    Automatic: Ascend will pick the winner for you based on your selected metric. Select a fallback version (required) — this version will be sent to remaining audience members if results are inconclusive after the test duration or date. Optionally, toggle Pick statistically significant winner. Send fallback version if statistical significance cannot be identified.
    ‍‍
11. Click Send‍
12. Once your broadcast kicks off, monitor A/B test and winner results in Reports.

‍

Create a scheduled A/B test 

1. Go to the Broadcast page and toggle A/B Testing to ON
2. Select your version A flow or draft your simple message
3. Click the version B tab and select your version B flow or draft your simple message for version B
4. If you would like to test additional versions click the + icon
5. Click Schedule your message and select when you’d like to broadcast your A/B test
6. Click Review Message
7. Select your audience split between message versions
8. Click Send

‍

Create an A/B test with pick a winner

Use these steps to schedule an A/B test that runs on a subset of your audience, then sends the winning version to the rest.

1. Go to the Broadcast page and toggle A/B Testing to ON
2. Select your version A flow or draft your simple message
3. Click the version B tab and select your version B flow or draft your simple message for version B
4. If you would like to test additional versions click the + icon
5. Click Schedule your message and select when you’d like to broadcast your A/B test
6. Click Review Message
7. Select Run A/B test on subset and pick a winner for remaining audience‍
8. In the Winner column, set the percentage of your audience reserved for the winning version. Then set the percentage of the audience you'd like to recieve the versions of your A/B test.
9. Choose your winning metric from the dropdown: Click Rate, Response Rate, or Call Rate.
10. Choose when to call the winner:
    
    Duration: Set a testing duration up to 72 hours.
    
    Date: Pick a specific date and time in the future to call the winner.
    
    Note: Ascend will mark approximately when your A/B test will complete. We reccomend that you select when to pick a winner after the A/B test has finished sending. Allow time for members to engage with your broadcast before a winner is picked.
    ‍
11. Choose how the winner is picked:
    
    Manual: You’ll pick the winner from the A/B test report once the test duration is up.
    
    Automatic: Ascend will pick the winner for you based on your selected metric. Select a fallback version (required) — this version will be sent to remaining audience members if results are inconclusive after the test duration or date. Optionally, toggle Pick statistically significant winner. Send fallback version if statistical significance cannot be identified.
    ‍‍
12. Click Send
13. Once your broadcast is scheduled it will appear on the Scheduled page.