Attend FREE Webinar on Digital Marketing for Career & Business Growth Register Now

A Beginner’s Guide To A/B Testing

A Beginner’s Guide To A/B Testing

AB-test-ss-1920 What is A/B testing?

A/B testing, or split testing is the process of running a simultaneous experiment between two variants of a web page to see which one performs or converts better. A/B test consists of creating an alternative page for a specific page and showing each of them to equal number of visitors. The two variants – variants A is commonly the existing design (the “control”) and variants B is the “challenger” new variants. When you complete a test, one that gives a better conversion rate, wins! Despite the name (A/B testing), the experiment can be conducted with as many pages as desired.

This is how the process works:


Assigning traffic weight in an A/B test

Traffic is randomly assigned to each page variant based upon number of test variants created – for example, if you are running a test with 2 page variants split the traffic 50/50 and if 3 page variants created then 33% traffic to each variant. To get best results it is advisable to divide traffic equally among the variants.

Some of the variables that can be considered for testing

  • The main headline – Page title (size or versions).
  • The call to action (CTA) – Size or versions.
  • Try a variation of photo used in the page – Preferably showing product or service offered.
  • Button design – Appearance of CTA (try making it bigger).
  • Button color – Test different color combination and find what works best.
  • Form length – For lead capture less number of fields that visitors needs to fill works good but if more data is required try running an A/B test with varying amounts of fields. This will help to understand what works best.
  • Content long copy or short copy – Product / service detailed information or information in brief with images and videos. Test different variants and find the winner.

A/B testing process

The following is an A/B testing framework you can use to start running tests:

  • Identify goal – Look for pages with low conversion rates or high bounce rate or low click rate or less number of form filling that can be improved.
  • Look for variables – Now, start looking for variables which might be reason for low conversion or high bounce rate. Once you have a list of variables, prioritize them and create variants one after another i.e. after testing one variable implement that change and then try next variable.
  • Create Variants – Using A/B testing tool, make the desired variants for the variable. This might be changing the color of a button, swapping the order of elements on the page, hiding navigation elements, or some other variant. Try and test page variants to make sure it works as expected before actual testing begins.
  • Start testing – Send visitors to these different variants. The variant that visitor visits is chosen randomly. Their interaction with each variant is measured, counted, and compared to determine how each performs.
  • Analyze results – Once your testing is complete, analyze the result. A/B testing tool will present the data from the experiment and show the difference between how the two variants of the page performed and decide the winner variant. If the new page variant wins apply those changes in the page or else keep the existing one.

Use the test learning experience and check if similar changes could be done on any other page also.

How long to run to test

How long marketer needs to run an A/B test depends on the amount of traffic they get. They might want to run tests for anywhere from a few days to a couple of weeks till a good number of visitors is exposed to different variants. Giving a test insufficient time can give inaccurate results. Test one variable at a time, and give each test sufficient time to run.

Why A/B test

A/B testing allows making careful changes to marketers experience while collecting data on the results. This allows to construct and learn better how certain variables of their test impacts user behavior. In another way, marketer can be proven wrong i.e. their opinion about the best experience for a given goal can be proven wrong through an A/B test.

More than just believing on ones experience and believing it to be correct, A/B testing can be used consistently to continually improve a given experience and improving goals like conversion rate. Testing one change at a time helps to pinpoint which variable changes had an effect on visitors’ behavior, and which ones did not.

A/B testing can help in decreasing spend on marketing campaign by testing ad copy. As it will help marketers learn which web page variant attracts more clicks. By testing the subsequent landing page, marketers can learn which layout helps them to achieve their business objective best.

Video: What is A/B testing? (explained in 1 minute)

Few A/B testing tools

A number of tools are available for A/B testing, with different focuses, price points and feature sets. Here are some:

  • Optimizely – Offers a simple visual editor that makes each element on every page editable. A free A/B testing tool from the search giant. A great option to get started, but lacks advanced features.
  • A/Bingo – It can measure any event, test for statistical significance and is extremely fast, meaning it has minimal impact on page load times. It’s simple to setup, simple install the plugin but requires programming and integration in code.
  • Visual Website Optimizer (VWO) – Is one of the easiest A/B Testing tools. Works across mobile, tablet and desktop websites, and is a simple one-time installation.
  • Unbounce – Let’s build, publish and test responsive landing pages, without any knowledge of HTML. Interface friendly and easy to use.
  • Google Analytics Content Research – A helpful and free A/B testing tool from Google.


  • Don’t test one variant one week and the second the next. When doing A/B testing, always test both variants simultaneously and split traffic between variants.
  • Don’t conclude too early as the results can be misleading.
  • Marketers should not take decision based on their gut feeling overruling test results. The winners in A/B tests can be surprising.


Accurate A/B tests can make a huge difference. By conducting A/B test and gathering data, marketer can figure out exactly which marketing strategies work best for the company and product. When marketer figures out that one variation works better than another and has evidence to back it up, it’s easier to make decisions and craft more effective marketing strategies. Carry out A/B test on the web pages of the site believing something can be improved always.

Image & Video Credit: Google, Youtube

Your Comment

Your email address will not be published.