All About A/B Testing and Conversion Rate Optimization

It’s no secret that quality marketing efforts take time, money, and other resources. If done the right way, your business can skyrocket and if not, well, your precious resources may be wasted. Unfortunately, psychics aren’t always accurate, so marketers must rely on other methods to ensure that their campaigns are effective and worthwhile. In this blog post, I’ll be discussing two of the most popular decision-making methods used by marketers: A/B testing and Conversion Rate Optimization. 

What is A/B testing?

A/B testing is, in its most simple form, testing two versions of something (whether it be a social media post, webpage, newsletter, etc.) to see which version performs better. It is most commonly associated with digital content, however, the concept has been in practice for over a century. It first began in the early 1900s by farmers to test out the effectiveness of their agricultural choices and was later adopted by marketers in the 1960s and 1970s (Harvard Business Review). They first used A/B testing to understand how effective their direct response campaigns were. For example, would a telephone survey or mail survey garner more responses? By the 1990s, A/B testing went digital and the rest is history. 

So how does A/B testing work? 

The first step of performing an A/B test is choosing what you want to be compared. As mentioned earlier, this could be anything from the font being used to the layout of a newsletter. Once the two versions are created and uploaded to the platform, then both versions are published. Visitors are then randomly shown one of the two versions. This randomization minimizes the effects of any other potential variables, such as type of device used and demographics of the user.

Once the A/B test has concluded, results will be provided, typically via software provided by the company. These software provide two conversion rates: one for the “A” version (Control) and one for the “B” version (Variation). They also provide the margin of error, so the reported numbers could look like this: 

Control: 17% (+/- 3.5%) Variation: 19% (+/- 1.3%)

These statistics mean that version A had a 17% clickthrough rate with a margin of error of 3.5%, whereas version B had a 19% clickthrough rate with a 1.3% margin of error. Marketing teams can then use these numbers to determine their tactics. Since version B had a higher conversion rate, one may want to consider implementing that version across the board. Of course, it’s important to consider the organization’s resources and whether or not the higher conversion rate is worth the trade-offs required. 

Now that A/B testing has been discussed in depth, it’s time to talk about another popular decision-making method: Conversion Rate Optimization.

What is Conversion Rate Optimization (CRO)? 

Conversion Rate Optimization, or CRO, is the practice of using data from your website to increase the amount of visitors who perform a desired action (Hotjar). CRO can reveal what works well on your website, as well as any barriers that users may be facing. It is represented as a percentage, which can be calculated with the following formula: 

How is Conversion Rate Optimization used? 

Compared to A/B testing, CRO is more of an involved process. Rather than having two options compared, resulting in statistics with a (usually) clear winner, CRO requires more abstract thinking. One must calculate the conversion rate for a specific variable and then decide whether or not that percentage is high enough for their liking. If the percentage isn’t high enough and changes are made, then the conversion rate must be recalculated regularly to determine if the change was successful. This requires a lot of trial and error, as well as creativity to determine which variables need to be altered. Some examples of variables that can be altered include: 

• Size of a button 

• Location of a forum 

• Typography

• Color scheme 

• Verbiage 

CRO can benefit many aspects of a business. According to Optimizely, here are some common conversion goals that companies have for their CRO efforts: 

• Pageviews

• Newsletter subscriptions

• Content engagement

• Product sales

• Shopping cart completion 

• Social shares

• Leads generated

Now that there’s a basic understanding of both A/B testing and CRO, I’ll compare and contrast the two methods. 

Similarities and differences between A/B testing and CRO

Both concepts help improve marketing activities’ impacts. They provide marketers with vital information about what is and isn’t working, however, they do slightly differ. CRO is a process, while A/B testing is a technique used within that process. A blog post on the website A/B Smartly worded this idea differently yet beautifully: 

This means that you can perform CRO without using A/B testing (although you should), yet A/B testing cannot be performed without also participating in CRO.

One more difference that I’d like to point out is that CRO is a concept that can be applied to multiple platforms at once - websites, social media, apps, you name it! A/B testing on the other hand, is limited to a single platform at a time. Of course, one could run multiple tests on multiple platforms, but the results only truly apply to the test in which the results originated from. Although the information is valuable, it is less applicable in other scenarios. 

All in all, it’s important to use both A/B testing and CRO to help your business accomplish its goals. Any form of marketing requires valuable resources, so it’s in your best interest to ensure that those resources are being used wisely. Thank you for reading and happy optimizing!