A/B testing with Adobe Target

Kieron Lawlor
1 Nov 2018

Adobe Target is Adobe’s optimisation solution hosted within the Adobe Experience Cloud platform.

As well as offering a powerful solution for A/B testing, it also delivers multivariate testing, personalisation solutions (allowing organisations to display relevant content), and mobile app optimisation, to deliver the best user experience.

A/B testing

If you’re not familiar with A/B Testing (also known as bucket tests and split-run testing), it’s a process commonly used by marketers to test two or more variants of the following across their website, e-mails or mobile apps:

  • Content – copy, imagery etc.
  • Navigation – how the user moves through the site or across a page.
  • Styling – the look and feel.

The purpose of A/B testing is usually to identify the best performing variant in terms of conversion, revenue or engagement. This could be the viewing of a specific webpage; time spent on the website, or anything right through to the completion of an order.

Tests / Experiences

A/B tests are handled in two main ways within Adobe Target:

  1. Using a single URL e.g. homepage.com where different experiences are created for each test by amending the content, navigation or styling for the page. This is the most commonly utilised A/B testing option.
  2. Using a URL redirect where each experience has its own dedicated URL, e.g. Experience A uses homepage.com and Experience B uses www.homepage.com/version2. URL redirects are commonly used when testing two differing purchase funnels, where the steps the customer takes between selecting a product or service and completing the purchase are substantially different, or where multiple versions of a page have been developed.

In both cases, Experience A is usually set as the standard / current version and is set as the control experience.

Adobe Target includes an online editor making it straight forward to create simple A/B tests to alter content – such as changing the heading or using an alternative hero image. The online editor also caters for basic style changes of a page – such as amending the background colour of a text box or changing the text colour.

The online editor also offers the option to change the HTML and CSS where more complex changes to the content, styling or navigation are required.

Targeting / Audiences

Adobe Target provides the option to specify audiences the A/B test is appropriate for. Tests can either be set to run for All Visitors, or only to run when the visitor is identified as being part of a specified audience.

Adobe Target includes several predefined audiences including the following:

  • Browser specific – Goole Chrome, Firefox, Internet Explorer, Safari etc.
  • Operating System specific – Mac OS, Windows OS etc.
  • Tablet Device specific – Using a tablet device
  • Search engine referrals – Arrived at the website via Google, Yahoo, Bing etc.

Audiences created elsewhere in the Adobe Experience Cloud are also available for use within Adobe Target.

Audiences can be combined and used as the targets for the test or set as exclusions.

For example, you may wish to run an A/B test on the home page to change the hero image but only if the visitor was not referred to the site via a search engine. In this scenario you would target All Visitors but exclude the audiences related to search engine referrals.

In addition to the predefined and Adobe Experience Cloud audiences. Adobe Target also allows users to create new audiences based on several additional rules, such as the pages the visitor has viewed and referring sources, such as specific campaign URLs.

Adobe Target also provides an option to create more complex audiences using Profile Scripts written in JavaScript. A typical scenario for using a Profile script is where an organisation offers an affiliate program with many participants. The Profile Script can be developed to group all visitors referred from affiliate URLs into a single audience.

Traffic Allocation

Allocating traffic during the A/B test can be managed in two ways within Adobe Target:

  1. Manual / user defined – This is set by the user when building the A/B test and is based on percentage splits between each experience, i.e. 50% of visitors to Experience A and 50% to Experience B. This is Adobe Target’s default method.
  2. Autoallocate – This option automatically allocates traffic to the best performing experience. Adobe Target will split the traffic equally between each experience and after a period of analysis, will allocate the traffic to the best performing experience, based on the goals for the A/B test. This option is commonly used when there is a revenue-based goal such as an e-commerce order or an online donation.

Goals & Reporting

Adobe Target has in built reporting capabilities to track the performance of each A/B test. Goals are split into three measures:

  1. Conversion – Viewed a specific page(s) or clicked a specific element(s)
  2. Revenue – Revenue per visitor or order.
  3. Engagement – Page views or time spent on the site.

A primary goal is set for each A/B test which is used in the auto-allocate traffic allocation mentioned above (if selected). Multiple additional metrics can also be added.

An example of a primary goal could be:

  • The visitor reached the order confirmation page;

and a supporting additional metric could be:

  • The visitor viewed the account details page.

Managing tests

When planning A/B tests, it’s important to avoid conflicts between multiple tests, and it’s worth considering any upcoming events or compaigns that could impact the measurement of success of each test.

How long an A/B test should run for varies depending on the number of experiences included in that test, the traffic volume and the type of allocation. In general, lower traffic volumes or tests with more than two experiences, usually require longer test durations.

Share

Related posts