Nearly three years have passed since Thomas Publishing Co. migrated its Thomas Register directory from a more static print product into ThomasNet.com, an online, interactive resource for engineers, purchasing managers and other industrial professionals. In that time, the ThomasNet.com team has developed a host of online marketing and educational communication vehicles to engage its target audience, including a blog, e-mail newsletters and an Internet newsroom. And since no direct marketer makes big, or even little, changes without testing, the online publisher also has been fine-tuning its process of site optimization.
This week, Target Marketing Tipline talks to Brad Mehl, vice president, marketing, about ThomasNet.com’s online testing strategies and how the results are being used to improve the user’s experience.
Target Marketing: What kind of testing is ThomasNet.com conducting to learn what works best online?
Brad Mehl: To put our testing activities into context, it’s important to share that our primary goal for ThomasNet.com is to connect industrial buyers with sellers. Our activities (and our testing) are focused on increasing conversions. Our definition of a conversion is an action that brings a buyer closer to a supplier; it’s an explicit expression of buyer interest. For example, conversions include views of a company profile, a click over to a supplier’s Web site and e-mails sent to that company.
We approach testing at two levels: 1) a “focused” approach; and 2) multivariate testing.
In our focused approach, we zero in on a specific part of our site or a particular user issue. This approach starts by analyzing user paths on ThomasNet.com, particularly on our most heavily trafficked pages. We try different usability solutions to improve the user experience for that part of our site. The focus approach is hypothesis-driven and encourages us to think deeply about the user.
For example, when a user comes to ThomasNet.com from a search engine, they typically land on a search results page on our site. Let’s say that someone starts on a search engine looking for “helical gears.” They click over to a page on our site with suppliers that make helical gears. That begs the question: “Do they know that ThomasNet.com has 65,000 other product categories beyond helical gears?” To address this, we tested a new link that tells users the range of content on our site. Early results are very encouraging.
In multivariate testing, we begin by identifying which variables on our site might make an impact during the user’s experience that can have a positive effect on leading a buyer closer to a supplier. We start broad—testing many elements on our site at once—and then we drill down to improve the pieces that statistically show promise.
Recently we tested 41 different variables on a page and tested thousands of combinations of those variables. It’s all about finding the optimal combination of variables on a Web page, the combination that delivers the most conversions overall.
TM: What are the advantages of using this two-pronged testing approach?
BM: Multivariate testing helps us to identify areas we may not have considered. Also, it reveals which variables on a Web page can make a difference. For ThomasNet.com, it’s very scalable approach because pages are changed dynamically by a system, so you get to learn quickly what works and doesn’t work.
The biggest advantage of the focused approach is that it is more human driven; we concentrate on the user’s experience. This takes into account user issues and encourages deep thought on our part about the user’s interaction with a Web page in the context of their task at hand. This type of testing can really help identify what users are trying to accomplish, not just what they are interacting with on your site.
TM: What site components are you testing?
BM: We concentrate on the pages that get the most traffic, including our homepage, search results page and the company profile page—where the basic information of a given supplier is displayed. Primarily, we have tested different approaches to navigation, or the best way to organize content on a page, and design, which is the best visual representation of that organized content. We’ve had a wide range of results and have seen increases of 5 percent to 67 percent, depending on the test.
One example of a focused test is [a change we made to] our e-mail form, which is filled out by users who want to get information from a company listed on our site. We found that by changing this form, we can increase the number of e-mails sent. What worked best was enabling the user to type their message first, before their contact info (as opposed to entering the contact info first). The thinking behind this suggests that users are more likely to commit to sending the e-mail once they have entered their comments upfront; requiring contact information up front interferes with this commitment.
TM: What has been the biggest lesson you’ve learned about leveraging the power of multivariable testing?
BM: Since so many things can be tested with a system-generated approach, there is a temptation to do too many tests. Tests need to be properly prioritized, and there needs to be a mix of both multivariate testing and a more focused, deeper approach on specific issues. We’ve found the best approach is to use tests in succession so that focused testing is used to shape the user experience, and the multivariate testing is subsequently used to refine the experience.
By using the testing methodologies together, we can get more out of each test than we could if we used them in isolation. Testing is about discovering improvements that will hopefully have a positive impact on a user experience. It’s important to keep the goal of each test top of mind. Ultimately, it impacts our bottom line when a user has a good experience.