E-commerce Link : Testing, Testing, 1, 2, 3

Optimize your results with multivariate testing

During the economic downturn, marketers had to become smarter at e-commerce. Savvy marketers tested e-mail campaigns, offers, on-site merchandising and targeting. Now, as the economy recovers, they must build on what worked to gain competitive advantage, grow their brands and increase their customer bases.

In any market condition, one of the most successful strategies is the use of analytics to help guide business decisions and make sure limited resources are focused on areas with the greatest potential for returns. Marketers now are taking testing to the next level. Many use A/B testing strategies to determine what works, dividing traffic equally between two different versions of a Web page to determine which one delivers the best results.

As marketers become more adept at using A/B testing, it may be time to explore a more effective variation called multivariate testing. Multivariate testing, as a methodology, lets site owners find out which zones of a Web page have the biggest impact on target metrics like orders, leads or time spent on the site. It also pinpoints which version of each page component produces the biggest lift in the metric of interest. With multivariate testing, each test looks at more than one part of the page and more than one variable at a time.

Key Considerations for Optimal Testing
It's clear that testing offers significant benefits and a much deeper understanding of customer preferences and behaviors. However, it requires time and attention to do it right. The following are some key considerations:

• Clearly define your objectives: Prioritize specific goals related to your business, and set your testing methodology around them. Common goals include factors like driving increased clickthroughs or improving navigation on search results pages. With your goals in mind, engage in a progressive process of continuous optimization, factor by factor.

• Understand your capabilities: Determine the amount of time your marketing staffs (and the IT people who support them) can commit to testing to assure a smooth process of refinement in areas such as page modifications, usability and site function.

• Creatively explore the unknown: The most successful and useful testing strategies measure factors in which the outcomes are uncertain. Marketers often suspect that they know what will work, but are often surprised with the results of testing. These unexpected results can drive improvements to functional areas within the site and dramatically improve business results.

• Implement informed site changes: Testing is a systematic method for allowing site visitors to define a site that is most effective for them. The benefit is that effective sites are the most profitable and create the most loyalty. Successful testing strategies result in a need to change merchandising strategies and site design. Execute tests with action plans in mind to implement changes to the site that reflect the tests results.

• Refine, test and learn: Testing is a practice that can positively impact marketers' businesses by implementing customer-centric approaches to site design and merchandising. Marketers who continuously measure and improve their sites produce fresh, effective sites that work for customers. The most successful marketers are open to unlikely outcomes and committed to making changes based on test results, which also means they are most likely to uncover new opportunities.

While all testing requires resources, multivariate testing programs require additional considerations:

• Resources—While more effective than simple A/B testing methodologies, multivariate tests can take significantly more time to design and execute. The most successful multivariate testing programs are continuous, so the resources needed to plan a testing strategy, create alternative content, set up and run the test, and share the results need to be part of the ongoing staffing plan.

• Technology—The technology required to run a multivariate test is more complex than what's needed for an A/B test. So, even if you built your own A/B infrastructure, you will need to move up to a multivariate testing application.

• Content—Multivariate tests require significantly more content than A/B tests. Marketing and site design teams need to be prepared to do extra work to develop this content.

The Benefits of Multivariate Testing
Gaiam, a large multichannel marketer, has used testing and ongoing site refinement to optimize sales. In one specific program this year, Gaiam tested several key zones on critical product pages. Informed by test results, the marketer focused on improving the product page for a popular product called the Balance Ball Chair. Gaiam was able to achieve positive results by analyzing shopper behavior and making appropriate changes to the site. Findings included:

• Most visitors zoomed in on the photo. Increasing the size of the graphic resulted in a better layout.

• Adding the product feature toolbar helped customers find the most searched upon information.

• Adding ratings and reviews reduced navigation away from the page as customers looked for feedback.

• Adding promotion data on the product page motivated customers to buy now.

• Adding related items increased average order size.

Jason Marshall, vice president and general manager of consumer direct at Gaiam, offers, "Testing gives us access to the data that helps us fine-tune our business. In a recent testing and refinement cycle, we were able to recognize important customer behaviors and make changes to our product pages that increased overall site conversion by over 10 percent."

If you have had success with A/B testing, now is the time to try multivariate testing to further optimize your site to increase revenues, grow your brand and expand your customer base.

Ken Burke is chairman, founder and chief evangelist of MarketLive Inc., an e-commerce technology services provider based in Petaluma, Calif. He can be reached at ken@mmlive.com, or visit www.marketlive.com/sitereview.

• MORE ON
• ONLINE MARKETING