Ecommerce A/B Testing: The Uber Actionable Step-by-Step Guide

ecommerce ab testing

I’ll be frank. A/B testing on eCommerce websites is overrated.

Unlike most ‘CROs’ preach, changing colors and tweaking fonts are only going to give you very marginal improvements in conversions and click-through rates. It takes quite a bit of trial and error to understand what creates a positive experience for users on eCommerce websites.

It takes even more effort to identify the standard elements on eCommerce sites and determining how they can be A/B tested to try and improve conversions.

You’re in for a treat today.

I’ve done all the work for you. I’ll show you 5 eCommerce A/B testing ideas that have the most potential to increase your conversion rates, and thereby your revenue.

By the end of this post, you’ll have a solid idea of what to A/B test on eCommerce sites, how to do it, and maybe even have enough ideas to offer this as a service to your eCommerce clients!

Understanding Pain Points

Ever wondered why A/B testing is so important for eCommerce websites? It’s because elements of the site directly impact sales that translate into actual revenue, unlike advertising, newsletter signups or page views.

The most important goal of A/B testing eCommerce websites is to identify pain points. Pain Points are simply elements in the sales funnel that put off visitors or gives a poor shopping experience.

A few examples can be confusing UI, too many CTAs, and even slow loading pages. All through this article, we’ll talk about how you can identify pain points that cause loss of visitors and users through A/B testing, and then work on solving them.

First, let’s take a look at the most straightforward, free, and simplistic way to find pain points on your store. I hope you’ve heard of Google Analytics 🙂 It’s free, easy to set up, and is one of the most robust analytics platforms out there. If you’ve already set up Google Analytics on your store, skip the setting up part below.

Setting up Google Analytics on your store

This is pretty basic, but if you’re a newbie, this should help you set up everything in a few minutes.

Go to https://www.google.com/analytics/web/#home and sign up for a Google Analytics account. It’s  pretty straightforward if you have a Gmail account already.

When you click Sign up, you'll be prompted to fill a form that looks like this:We're setting up the account for our website, so choose Website. Next, just enter any Account Name and Website Name of your choice.

Then enter the Website URL in the correct format. Jus so you know- you can have a whole bunch of accounts in a single Analytics account.

Just chose which one you want to add your property (the Analytics property of your store), and click on Create new property.

Choose a suitable niche from the dropdown for Industry, and for Reporting Time Zone, just choose your own time zone. Then click on “Get Tracking ID”. Copy the code in the box below Website tracking.

Now you've got to add this code to all the pages on your site that you want to track. It's always a good idea to delegate this over to your designer or developer.

But it's simple if you use WordPress (with WooCommerce), Shopify or Magento. All these CMSs have page template files where you add this code.

Just make sure you add the code in the header area so that Google Analytics tracking code or pixel fires as soon as the page loads. Just so you know, you'll need a few months of data after you set everything up in order to proceed. There is no way around this.

Just so you know, you'll need a few months of data after you set everything up in order to proceed. There is no way around this.

Any type of testing requires a hypothesis or theory:

The eCommerce A/B Testing Hypothesis

In simple words, this is the step by step process we use for all A/B testing processes. And it's quite straightforward:

  1. The stats or data imply something
  2. We predict that (this) change will cause (this) desired effect
  3. We identify what metrics or KPIs to measure the effectiveness of the change or the validity of the prediction

The Stats or data for the implication

I'll be frank. I don't like to reinvent the wheel. There are tons of proven stats, data and suppositions on changes that affect conversions.

For example, the average shopping cart abandonment rate is a whopping 65.23%. And that's people who come to your website, add a product to their cart and then don't buy.

And data shows that 44% of such cases arise because of high shipping costs. Now, this is the statistic or data that we have.

We can confirm this from our Google Analytics data too.

For the sake of this example, let's take a look at the analytics of the real Google Merchandise store, to which Google has given public access.

After you get access to the account (instructions at the above link), go to Shopping Behavior in Conversions.

And set the date for as many months as you can (I prefer six).

As you can see, a whopping 16,753 sessions end in cart abandonment, and 6,345 sessions end in checkout abandonment.

We just identified a pain point here. Note that stats or data need not be just about pain points on your website, they can also be about features that improve user experience, like adding trust or award badges (Express Watches leveraged this technique to increase their sales by 58.29%), or offering live chat assistance (44% of online buyers would like to have online chat assistance during shopping, according to this Forrester study).

The Prediction

Based on the above data, we predict that shopping cart abandonment rates will decrease and check out rates will increase if we offer free shipping for users. Obviously.

The Metrics or KPIs to Measure

Now that we have the stats and predictions in place, we need to identify the metrics in analytics that will validate this prediction when we run the test.

Since we're looking to increase the percentage of users who add products to their cart and then buy them, the KPI we should be measuring would be the percentage of sessions with transactions.

Running the A/B Test

There are two different types of A/B testing tools.

Some are client-side, and some are server-side. In simple words, client-side tools make use of Javascript overlays on websites so that the browser makes the same page look different to the user. Think of adding some ‘makeup' on your web page.

Server side tools send different versions of the webpage altogether. They are much more robust, but require developer support, making them unsuitable for small teams (like that of mine).

Client side tools are easy to use, and most of them are simply drag and drop. VWO, Optimizely, Google Optimize and Adobe Target are all client side tools.

And the setting up is as easy as it gets: all you need to do is add a Javascript snippet on your website (similar to the Google Analytics tracking code) and you're set.

But we all like to be spoon fed, huh?

Let me show you how to set everything up step by step using the free Google Optimize tool.

Running an eCommerce A/B testing campaign using Google Optimize

First, signup for Google Optimize account here.

After you've signed up and given all the account permissions, you'll land on the Dashboard, from where you can start creating ‘Experiments'.

Go ahead and fill in all the details for the experiment:

Enter any name for the experiment, and the URL of the page we wish to A/B test. In our example, it's the checkout page (https://shop.googlemerchandisestore.com/basket.html).

Choose A/B test, and then click ‘Create'.

The next step is to link your Google Analytics property to the experiment. Click on ‘Go to container page' in the right sidebar.

On the Container page, click on ‘Link Property'.

Select the property associated with the website you plan to run the A/B testing campaign on from the drop-down in the popup that appears, and click on ‘Link' (Do note that you need edit access to the property you're linking with Google Optimize).

Next, you'll be prompted to add a short code snippet to your site to run Optimize experiments. Click on ‘Get Snippet'.
Now you'll be shown a shortcode that looks like this (step 2):

It's pretty simple- just insert that code into the Google Analytics tracking code as shown. Don't worry about the code being as high as possible in the header. Just make sure there isn't any JS that loads above it in the header.

Then click ‘Next'.

Now, you'll be prompted to add another code:

Remember how Google Optimize is a server side tool and uses a Javascript overlay to show two versions of the same page. Well, this code is just to make sure that the users aren't shown one version before the javascript code loads. Just paste it into your page template (or shopping cart page HTML) just above the Analytics tracking code in the header.

After you've added the code snippets, return to the Experiments page, and click on our ongoing experiment from the list.

Setting up the experiment is done in three steps:

  1. Creating one more variants
  2. Selecting a primary objective
  3. Setting targeting by creating a URL or path rule

Let's start by creating a variant.

You can preview the original version or make edits right from this page.

Try-Shopify

One of the most interesting and useful features of Google Optimize is the Google Optimize chrome plugin that allows you to create drag and drop variations and make edits easily. Go ahead and install the plugin right now.

You can click on original to make changes or edits to our original page version (version A).

Now let's create the variant version (version B). Return to Google Optimize and click on “NEW VARIANT”.

Name this version as you'd like and click on it to open up the checkout page (our example) and make edits or changes.

To edit or change any element, just select an element, and click on the icon marked in red.

This opens up a popup where you can change the font and size, alignment, color, background, border, etc.

For example, since we're planning to offer free shipping and test how it could helping reducing dropouts from the checkout page, let's add a line below the pricing that says ‘Your order qualifies for free shipping to US and Canada'.

Click on ‘Edit Element' in the popup, click on ‘Edit HTML', and enter the text in the popup editor and click ‘Apply'. That's it. Click on ‘Save' to save all the changes you've made to the variant version.

With a bit of HTML coding skills, you can alter entire designs and add new visual elements as you like. And you can create as many variants as you'd like in this manner.

Now that we've created a variant, let's move on to the next step: setting an objective.

Below the variants table, you can see a table for Objectives and Targeting.

Select an Objective from the dropdown:

In our case, since we're trying to reduce dropouts (or bounces) from the checkout page due to high shipping charges, the objective is Bounces.

You can also add any secondary objectives if you'd like.

You can then add simple hypotheses- something that looks like this: “We plan to decrease bounce rates from the checkout page by offering free shipping”.

Then click on to the third step: Targeting (the other tab in the same table)

In the Targeting tab, there are a lot of settings that you need to configure.

Percentage of visitors: This is simply the share of the net traffic to that page that you want to run the experiment on. This is useful in cases when running the experiment to the entire traffic bucket might cause additional overhead charges or expenses. For example, offering free shipping for all orders isn't probably a good idea at the start of an experiment.

So let's set this to 5% so that the variations are shown to only a small percentage of shoppers.

Weighting of visitors to target: This is the percentage of visitors (in our case, 5% of all traffic to the page) to which each variant is served.

W usually set this as 50% for each variant, but in case you have, say, 3 variants of the page, you can set the percentage share of traffic as 33.33%.

After these, we have the ‘When' settings that determine when the experiment is shown:

Evaluate on: This setting can be used to decide when the experiment is loaded, on page load or after a custom event. This can be helpful for experiments that involve, say, testing what's shown to users after a social share (after a custom event).

Additional Conditions: These involve URL matches and additional targeting settings.

‘URL matches' is where you configure where to run the experiment. And for those of you who've been wondering all the while how we set dynamic URLs that are very common on ecommerce websites, here is where you configure it.

Just set the URL path as URL contains https://shop.googlemerchandisestore.com/basket

In our example, the target page was https://shop.googlemerchandisestore.com/basket.html

This ensures that the experiment is run for all dynamic URLs like https://shop.googlemerchandisestore.com/basket.html?vid=20160512512 that are common on the Google Merchandise store. Then click ‘Save'.

Other additional targeting settings include a whole range of demographics and other targeting options:

I know that this looks too complex for you.

But here's a simple example of how we can use this.

Since we plan to offer free shipping only to the US and Canada, don't you think it would be a good idea to target users only from US and Canada?

Let's set the Geo settings to Country equals United States and Canada.

The click on ‘Add' to save that targeting option. We're all set with Targeting.

Note: Just before you proceed, ensure that the variant is displayed properly on all device sizes.

Now click on ‘Save' to save the Variants, Objectives, and Targeting settings. Then click on ‘Start Experiment' to kickstart your A/B testing campaign.

That's it. You just set up your first eCommerce A/B testing campaign.

That wasn't so difficult, was it?

Now that we've learned the basics of setting up A/B testing (yes- that was just the basics), let's take a quick look at the more advanced and scalable ways to do this.

Multivariate Testing and Bandit Algorithms

First things first. Consider MVT only if you get a lot of traffic. You might have guessed it from the name- MVT involves

You might have guessed it from the name already- MVT involves testing multiple combinations of elements on a page to find the best combination of elements that meets your conversion goals.

Sounds confusing?

I'll explain!

Suppose you need to test your header image, logo, sidebar and footer and want to find the best combination of these elements that drives the maximum number of clicks to your sidebar ads. This is nearly impossible with normal A/B testing, but a fairly common goal with multi-variate testing. Almost all A/B testing tools, including the free Google Optimize, offers you the ability to run Multi-variate testing campaigns.

Bandit algorithms are simply the automated way to run large scale campaigns and automatically reduce ‘regret'. ‘Regret' is the revenue or conversion lost due to a low converting variation.

In simpler words, suppose we run an A/B/n test for a page that receives a considerable amount of traffic. You can use a bandit algorithm to automatically set the winning variation (the page that has the most positive conversion results or goals).

Bandit algorithms are most commonly used in large scale multivariate tests where real time updation based on results is necessary to mitigate revenue loss.

Now that we've talked about how to setup and run A/B testing campaigns, let me give you some inspiration to run your own campaigns.

5 Proven A/B Testing Ideas to Increase Your Bottom Line 

1. Make the search bar as prominent as possible

Okay, I”m biased about this one. But let's be frank. How many times have you come across an online store and wished that there was a search bar?

Of all the online stores I've come across other than Amazon, I hardly remember any that had prominent search bars.

Amazon kills it in this regard.

Try testing your homepage or main category pages with a variant that has a prominent search bar. Experiment how it increases the views to other product pages and reduces bounce rates.

2. Add Trust Badges

With the hundreds of thousands of websites using this tactic, this should have been #1 in this list.

Ever wondered why more and more websites are prominently displaying the McAfee secure widget?

It's a simple psychological trick: by associating a well known trusted brand with yours, you gain a little bit of that trust. And it's an age-old trick.

Here are the most used trust badges:

Of these, McAfee Secure and VeriSign are regarded as the most trustworthy ones.

3. Add Product Videos

Besides providing an additional channel for promotion, this is a cool tactic to give users a clear cut idea of the products you're selling.

Zappos uses this tactic brilliantly and has videos for almost all their products.

Appliances Online found that users who watch their video reviews are 120.5% more likely to buy the product, spend 9.1% per order and also spend 152.7% longer on the store.

Just shoot a quick product video of your inventory and display them on the product page. Try testing this variant against the original.

4. Offer Next Day or Free Delivery

I know I already talked about this one, but I couldn't have not included this.

Stats are stats. This comScore study shows that a whopping 47% of customers just abandon the cart if they see that they're being charged extra for shipping.

While offering free shipping might be expensive, it's still a good idea to test a variant that offers free shipping (like we did in the above example) and see if you can increase your average order value enough to make it profitable. Nothing beats free shipping!

5. Add a Sense of Urgency

If you're thinking that displaying timers and offer expiration notices on product pages are done only by rip-off affiliate marketers, think again.

Even Amazon does this:

Industry practice is to entice users with next day or free delivery if they order within a said time period. You can also experiment by offering promo codes and freebies along with the product.

That's all from me! Here are several more ideas if you'd like to keep reading.

Now- wasn't that actionable advice?

Go run a simple campaign with any of the ideas above. And with Google making Google Optimize free for everyone, there's nothing stopping you! Share your results in the comments below!