Split tests
Test one variable on an ad set or an ad, launch and end the test through approvals, and read a verdict computed from your own figures.
A split test runs two to four versions of an ad set or an ad that differ in one variable, then names a winner once the difference is reliable.
Two kinds of test
| Meta split test | Tracked by Adrails | |
|---|---|---|
| Starts from | A Meta ad set | An ad, on Meta or Google Ads |
| How it runs | Meta divides the audience between the ad sets, so nobody sees two versions, and each ad set spends its own budget. | The ads run side by side in the same ad set. The audience is not split and the budget is not divided: the platform decides who sees which ad. |
| Measured by | Meta's study and Adrails | Adrails, from the launch |
| Needs | The campaign uses Ad set budget without budget sharing, the ad set has its own daily budget, and the account belongs to a Meta business portfolio. | An ad Adrails can copy. |
A Meta split test gives the cleaner answer, because each person sees one version. A test tracked by Adrails is lighter to set up, but the platform may show one version more than another.
What can vary
One variable per test:
| Starting from | Variables |
|---|---|
| A Meta ad set | Audience (age, gender, countries), Placements (none checked means Advantage+ placements), Bidding (bid strategy and its amount or minimum ROAS) |
| A Meta ad | Ad text (primary text, headline, description), Creative when the ad has one image or video |
| A Google responsive search ad | Headlines and descriptions (3 to 15 headlines of up to 30 characters, 2 to 4 descriptions of up to 90) |
| A Google responsive display or Demand Gen image ad | Creative |
Carousels, dynamic creative and existing posts cannot be tested. Google ad groups cannot be tested from Adrails: test the ads inside them instead.
Create a test
- In Campaigns, open an ad set or an ad and choose Split test.
- Name the test, choose What varies, and choose what it is Decided on: CPA or Conversion rate.
- Set the duration, from 4 to 30 days. The default is 14.
- Fill in the variants. A is the original as it runs; add B, and up to D.
- For a Meta split test, split the daily budget between the variants. Each gets at least 10%, and the shares add up to 100%. The original keeps its budget; the new ad sets get what makes the split.
- Choose Create variants.
The new ad sets or ads are created on the platform, paused. Nothing spends until the launch is approved.
Launch
On the test's page, choose Launch test. Adrails checks the platform first: the campaign and the ad set must be active, and the variants must be ready to switch on.
The launch is one approval, waiting in Priorities on Home. Approving it switches the variants on and, for a Meta split test, creates Meta's study. If the platform changed since the launch was prepared, the approval is refused and you prepare it again.
Before launch, Cancel test stops it. What was created stays on the platform, paused.
Follow the test
The test's page shows its status (Not launched, Running, Winner found, Ended), the day it is on, how many conversions it has and how many more it needs, each variant's share and budget, and the same metrics, verdict and day-by-day chart as Compare. Only the days since the launch count.
Adrails checks running tests about once an hour.
How the verdict is decided
- Conversion rate compares conversions per click (per link click on Meta), or per impression when a variant has no clicks.
- CPA compares each variant's conversions with its share of the spend.
- The leader is tested against every other variant at 95% confidence, corrected for the number of variants.
- No winner is named before every variant has at least 10 conversions.
When the leader is reliable on the metric the test is decided on, the status becomes Winner found and the test appears in Priorities. When the end date passes without a reliable leader, the status becomes Ended and nothing is paused.
End the test
Once a winner is found, choose End test: pause the others. Adrails prepares a pause for each losing variant, waiting in Priorities. For a Meta split test, approving one of them also stops Meta's study.
Watch for
- Creating the variants needs no approval: they are created paused. Launching and ending do.
- A Meta split test needs the campaign and ad set switched on before launch.
- A launch approval expires after 24 hours.
- Only owners and administrators can create or cancel a test and approve its launch and end. Any member can prepare the launch or the end for them to approve.