CRM Special Report: Untapped Gold
How to Extract Value From Open-ended Survey Responses
There is gold in open-ended survey responses. As opposed to close-ended questions where a respondent is given a finite selection of answers (e.g. very satisfied, satisfied), open-ended survey questions have no specific answers, and respondents are free to share their thoughts.
Direct marketers conducting surveys include open-ended questions to gather prospects’ or customers’ opinions on offers; acquire names; assure quality control; or, sometimes break the ice.
The words respondents choose to answer open-ended questions offer valuable insight that can be acted on to improve direct marketing results.
A Wealth of Information
Fairytale Brownies, a mail-order company based in Chandler, Ariz., understands the value of seeking customer opinion through surveys. The company inserts in each package of its brownies a survey card that asks an open-ended question such as, “How did the brownies taste?”
The company initially set up the survey for quality control purposes, but soon learned the responses, particularly to the open-ended question, provided it with useful information such as new product ideas. The survey also serves as a vehicle for building customer relationships. Respondents are asked if they would like a representative from Fairytale Brownies to call them, and if they say “yes,” someone does.
Although Fairytale Brownies learned the value of open-ended survey responses, many surveyors choose to ignore open-ended questions because, unlike multiple-choice questions, open-ended answers are difficult to analyze.
The standard method of analyzing open-ended survey responses is to assign codes (usually numbers) to the different responses. The person analyzing the open-ended responses develops a set of response categories that adequately represent the answers given. Then the number of answers that fall into each category is determined. For example, if survey participants were asked, “What features of a product most satisfy you?” their answers probably can be represented on a continuum of response options. Coding typically is done after the survey is completed, because the list of possible answers can only be generated after the survey has been conducted. There are, however, cases when the coding can start before the survey ends. This can be done manually or by using one of a number of software programs that can assist in accelerating the coding process. Coding is critical, because it enables the responses to be statistically analyzed.
Even with technology, analyzing open-ended survey responses still is a labor-intensive and complex process. Human speech, by its nature, is unstructured. Many words have several meanings, and there are countless ways to express a single idea. It is difficult to manually detect the semantic closeness of every possible pair of sentences as well as take into account possible spelling mistakes.
There are new technologies available that assist in categorizing text responses. New solutions combine manual techniques with advanced linguistic technologies designed to extract and classify key concepts from within survey text. Natural Language Processing (NLP) technologies analyze text as a set of phrases and sentences in which the grammatical structure provides a context for the meaning of the text response.
This linguistics approach equates terms that are used in similar contexts. For example, NLP would equate terms like “executive,” “manager,” and even “mgr,” if they are used in similar contexts, whereas a non-linguistics-based solution may not equate manager with executive if it weren’t specifically stated. In addition, NLP technologies can interpret the tone of text. A customer comment about “new phone” may be distinct from a related concept, “new phone ASAP” response. The understanding of context cuts through the ambiguity of text, making linguistics-based text analysis the most accurate approach.
Technologies, like linguistics-based analytic software, are making it easier for direct marketers and researchers to pan for the gold in open-ended survey responses. The ability to personalize offers and better understand customers is a key factor in successful direct marketing. And, the best way to find out what is in customers’ heads, is to ask them.
Jason Verlen is vice president, survey applications business center at SPSS Inc. He can be reached at firstname.lastname@example.org.