Advertisers looking to improve their performance have long used A/B testing to improve their ads. It is well understood how this is done with Expanded Text Ads (ETAs).
But recently, Google has announce ETA will be finalized, making responsive search ads (RSAs) the main text ad format from July 1, 2022 onwards.
Time to revisit our ad testing best practices.
What is A/B ad testing?
Even novice advertisers who don’t have a clear goal for ad testing may have already done a significant amount of A/B ad testing by taking advantage of the ability to add multiple ads to an ad group.
This is done so that Google can show you the best performance more often.
Optimized rotation is, after all, the default setting for ad rotation.
-
Screenshot from Google, December 2021
More advanced advertisers usually choose the option to rotate ads indefinitely and use their own methodology to pick winners and losers. Instead of focusing on CTR, advertisers care more about conversion rate.
But focusing solely on conversion rate ignores the possibility that a high converting rate ad will get very few clicks due to a low CTR, thus leaving the advertiser with fewer overall conversions.
By combining CTR and conversion rate into a metric called Conversions per impression (CTR * Conversion Rate = Conversion/Imp), advertisers get a better measure of which ad is better at achieving one of their goals (which is to maximize conversions).
In the following example, you can see a traditional A/B test between RSA and ETA.
Based on performance data from Optmyzr’s recent RSA study (Disclosure: My Company), the average RSA has a 10% better CTR but a 20% lower conversion rate than the ETA in the same ad group.
Combining these two measures, RSA is 12% worse than ETA at converting an impression into a conversion.
-
Photo by the author, December 2021
This finding helps explain why some advertisers remain reluctant to use RSAs comprehensively: by traditional A/B metrics, they appear to perform worse.
Why traditional A/B testing methods don’t work with RSAs
But just like conversion rate alone paints an incomplete picture about which ad performs best, conversion/puck provides an incomplete picture when RSAs are added into the mix.
Here’s the problem. RSAs qualify for a lot more impressions than ETAs. Based on our study, RSA is so much better at getting good ad rank that it can show it for about 4 times as many queries.
All else being equal, RSAs get 4 times more impressions than their ETA.
RSA ads get a lot more impressions than ETAs because they get better ad rank for more inquiries.
-
Photo by the author, December 2021
The impressions per ad metric was calculated by taking the impressions of a given ad type and dividing by the number of ads of that type.
In the study, we separately analyzed ad groups containing only RSAs, those containing only ETAs, and those containing both types of ads.
The number in the graph above is based on more than 157,000 ad groups that contain both ad formats. Even when we looked at ad groups with only one ad format, ad groups with RSAs got more than twice as many impressions per ad group as ad groups with only ETAs.
So our old assumption that RSAs and ETAs compete for the same set of impressions based solely on targeted keywords in the ad group is wrong.
Since RSAs get a lot more impressions than ETAs, even with slightly worse performance on the conversions-per-impressions metric, they are still likely to bring in more conversions.
RSA A/B tests should include impressions in the analysis
A more accurate way to identify a winning ad when RSAs are part of the mix is to incorporate the impressions into the analysis as shown in this graph.
-
Photo by the author, December 2021
Any A/B testing including RSA should include impression data so you can measure it incrementally.
As we explained before, RSA has a 12% worse conversion rate per impression compared to ETA. We have conservatively added double impressions for RSA which drives 76% more conversions.
So RSAs are the real winner if you care about getting a lot of conversions.
When converting your A/B tests from ETAs to RSAs, it’s important that you include impression volumes so you can get the real increase for your RSAs.
Side noteConversions are usually not the real goal for advertisers as earnings are also important. Since more advertisers are now using automatic bidding, this article will not cover how to ensure that these additional conversions are profitable.
This should be handled automatically by setting the right goals and entering the correct conversion data into the ad platform.
Even if you’re only A/B testing ETAs, keep impressions in mind. Historically, this was more because we wanted to make sure that any ads we compare had an equal chance to prove themselves.
If one ad is much newer than the other, it will have much lower impressions, thus invalidating the test. Or if an ad is only shown on weekends and other weekdays, there will be a similar difference in impression volume which could invalidate the test.
Now, even when two RSAs are running on the same days and have been active for a long time, they may still generate impression inconsistencies because one is better than the other at qualifying itself for more auctions.
Make sure to consider impression volume in all of your ratings.
Now that we’ve established a better way to measure results, let’s look at some ways to find what to test when creating new ad variations to challenge existing ads.
Ad strength indicates whether or not you’re following best practices
As you write a new RSA, Google immediately provides feedback in the form of an ad strength indicator.
Ad Strength is a best practice score that measures the relevance, quantity, and variety of responsive search ad content even before your RSA is served.
According to Google Slides, each increase in ad strength corresponds to approximately a 3% increase in clicks.
So going from “poor” to “average” should result in about 3% more clicks, and going from “medium” to “good” will result in about 3% more clicks.
-
Screenshot from Google, December 2021
Ad strength is a best practice metric and should only be relied upon while creating a new RSA. But the main point is that this is the result of “best practice” and has nothing to do with actual performance.
Just as you can have a keyword with a low Quality Score that leads to valuable conversions for your account, you can have an ad with poor ad strength winning over other ads with better results.
Try to follow Google’s instructions to start your next quiz with the best odds of winning.
But don’t get stuck trying to get the perfect score. Actual performance will be reflected in the asset performance aggregation ratings, so let’s take a look at those next.
Asset performance labels reflect your performance
Once RSA has accumulated around 5,000 impressions at the top of search pages over the past 30 days, asset labels give you guidance on which pieces of text are performing well and which you should consider optimizing.
The Google Publish A helpful table of what the different performance aggregation labels mean:
-
Screenshot from Google, December 2021
Most importantly, replace any assets with a ‘low’ rating and also replace assets that haven’t gotten any impressions after more than 2 weeks.
Remember that what you see in this asset report is based on actual performance. It is important to optimize based on this rather than ad strengths.
test ideas
Not many ads will get enough top impressions in the 30 day period to show performance data, but there are still ways to improve if that’s the case for you.
Remember that Google allows up to three RSAs per ad group and allows certain pieces of text to be pinned to different positions in the ad.
With this, you can create some useful experiences.
-
Screenshot from Google, December 2021
Advertisers can now pin multiple assets to one place in an RSA ad.
The first one to try is based on messaging.
Create multiple RSAs, each with a different base message. The key messaging elements of any ad, be it an RSA or an ETA, consist of the company name, a unique value proposition, and a call to action.
A value proposition and a call-to-action are some of the best things to try since your brand is your brand and for most advertisers, there isn’t a lot of room to get creative with that.
For example, the first RSA could focus more on the large selection your site offers, while the second could focus on another value proposition such as how high your site is rated by previous customers.
Then when you use your own A/B testing process, you’ll find which message resonates the most and what leads to more conversions.
A second experiment takes messages that worked well and tests them in different situations.
You can do this by pinning a group of assets to different locations such as a specific title or description.
Google now allows multiple assets to be pinned to a location, so even if you have 4 assets for your value proposition, you can pin all 4 assets to the desired location. Now when you run your A/B test, you may find that the value proposition works better when pinned to the Heading 2 position.
-
Photo by the author, December 2021
Installations
Whether or not to install is one of the biggest debates in RSAs. After Sunset You Can Only Create RSAs Starting July 1, 2022, advertisers can pin one piece of text per site to effectively convert an RSA to an ETA.
But this may challenge the whole purpose of RSAs which is to get the machines to help create the best possible ad for each user on the go, making the ad more relevant and therefore eligible to appear for more searches.
There are industries where installation is a legal requirement. But besides that, here’s a breakdown of your installation and whether it helps (or hurts) performance.
-
Photo by the author, December 2021
Pinning is ultimately about advertiser control. And what the data shows us is that the device drives the best CTR when advertisers allow the algorithms to do their work without interfering. CPC and CPA are also lower in this case.
But when advertisers want control, they’re better off giving the device at least two different forms of text for each pinned placement.
Pinning a single piece of text to a site has the most negative impact on CTR, CPC, and CPA.
Installing multiple assets seems to provide a good balance between control and performance.
Title Variations
The ad strength indicator takes into account the number of variations you provide for your titles and descriptions. Since coming up with 15 addresses can be daunting, we analyzed whether it is really worth adding more than 2 addresses per RSA.
-
Photo by the author, December 2021
What we found is that ads with more headline variations drive more conversions per ad. So when A/B testing RSAs, more addresses are actually better.
conclusion
If you remember just one thing from this post, it’s that the basic assumption about ad testing needs to change. Impressions are also now highly dependent on advertising, and are no longer just ad group keywords.
When A/B testing RSA ads, it is crucial to consider this potential difference in impressions and try to find the ad that generates the most increase.
Over time, we expect Google to provide more metrics for RSA assets and this will help all advertisers build on the optimization techniques covered in this post.
More resources:
- 7 Powerful Benefits of Using PPC Ads
- This is what a universal search ad looks like
- Top 10 Pay Per Click Trends To Know In 2022
Featured image: Jane0606/Shutterstock
