Readers respond to “Warren Buffet’s Five Secrets of Success,” published July 6 2006, that examined how the world’s second-richest man does business.
Garfinkel’s piece on Warren Buffett’s secrets of success was excellent. I’d like to add a couple of take-away points. Dave Thomas of Wendy’s said in IMPRIMIS magazine about a decade ago, and I paraphrase from memory, “It doesn’t do one much good to be the richest man in the cemetery.” My own pithy aphorism is, “He who dies with the most toys, is dead.” Lastly, whether or not one is religious, I share three hard sayings of Jesus: “It is easier for a camel to go through the eye of a needle than for a rich man to enter the kingdom of God” (Mark 10:25). “You cannot serve God and mammon” (Matt 6:24; Luke 16:13). “And I tell you, make friends for yourselves by means of unrighteous mammon, so that when it fails they may receive you into the eternal habitations” (Luke 16:9). I think Messrs. Gates and Buffett are wise to distribute their wealth far and wide and put it to effective use. They stand a chance of beating the odds.
(To David Garfinkel)
Hi, David: I know Denny and am impressed by anyone he chose to do a column, and yours was, I thought, excellent. The late David Ogilvy was first to tell me about Mr. Buffett—and the one thing you never mentioned is that he (Buffett) is a wonderful writer; his annual reports are collectors’ items. More strength to your quill (I am stuck in the 18th century). Best wishes!
A reader responded to “Before Changing Your Business Model, Consult a Direct Marketer” published June 29, 2006, that discussed how Zales changed its business model without testing. I quoted a business rule by direct marketing guru Axel Andersson and it generated some confusion. Here’s an exchange between Phil Langsdorf and DH.
*”For a test cell to be a clear winner, the number of responses must be more than double the square root of total orders.”
This is one of your takeaway points from your June 29 newsletter. Are you sure you copied it correctly? I did an A/B test and got almost the same number of orders (171 in A, 178 in B); Total orders is 349. Square root is 18.7. Double the square root is 37.4. Response in B is 4 .75 x that, but Z test says there is only a 30% confidence that the tests are different. Is the rule supposed to be that the difference in the number of responses must be more than double the square root of total orders?
Thanx for writing. Example: You have two tests that bring in 100 orders. The square root of 100 is 10. Thus if test A brings in 60 orders and test B brings in 40 orders, test A is the winner by 20 orders or double the square root. But be cautious: Keep in mind that the larger the numbers, the more reliable the result. Cheers.
Langsdorf replies to DH:
Thanks for the quick reply. I agree with your example, but don’t think it matches the rule as written. It would match, “For a test cell to be a clear winner, the increase in the number of responses must be more than double the square root of total orders.”
DH replies to Langsdorf:
I am not sure I get this. Could you explain? Thanx.
Langsdorf replies to DH:
No big deal. In the rule as written in your column, “the number of responses must be more than double the square root of total orders”. The number of responses in the example you sent me was 60. The key is that the winning margin is 20, which is 2 x the square root of total orders. In another test, the winning cell may have had 60 responses, but the other side of the test may have had 59. Thinking like a lawyer or mathematician, the test met your rule, but clearly did not win. Enough. Thanks for the newsletter. I enjoy it.
DH to Langsdorf:
Sorry to belabor this thing, but if the winning cell has 60 and the other cell has 59, total orders are 119. The square root of 119 is what? I cannot do square root any more, but it seems to me that the equation changes. Or am I losing my mind? Cheers.
Langsdorf to DH:
Only people with calculators can do square roots. Sqrt 119 is 10.9. Rule you quoted says: “For a test cell to be a clear winner, the number of responses must be more than double the square root of total orders.” —Axel Andersson
Double 10.9 is 22. 60 is more than 22. Therefore, by the rule as written, the test is “a clear winner.”
But it isn’t. My guess is the rule should be: “For a test cell to be a clear winner, the increase in the number of responses must be more than double the square root of total orders.” The results would have to be 71 to 48. That would be a clear winner.
In the 48 to 71 example, the Z test (assuming sample size of 2000 for each test) would be 2.14, which gives a one tail confidence level of 96.8%. That is a clear winner.
And I think either of these rules assumes that the mailing size for both samples was the same. If you mailed 5,000 to get 71 and 1,000 to get 48, the 48 is the winner.
In short, the rule is too short.
A potential subject for a newsletter article is how people make decisions when the difference is not that dramatic. At 71 to 48, I wouldn’t ordinarily even be doing the math. I’d probably be spending time to be sure there wasn’t an error in the data.
To go back to your article, if Zales had done the test you proposed, I seriously doubt that the difference in sales would have been 71 to 48, or an almost 50% increase. In fact, in 2005, sales were up slightly, but not what they expected—and had led Wall Street and the board to expect. From their press release at the time:
“Net earnings for fiscal year 2005 were $106.8 million, or $2.05 per diluted share. For the prior fiscal year, net earnings were $106.5 million, or $1.99 per diluted share. For the fiscal year, total revenues increased 3.4% to $2.383 billion, compared to $2.304 billion for the prior fiscal year. On a comparable store basis, sales increased 0.3% for the year.... “The increases to sales and earnings in fiscal 2005 did not meet our expectations, particularly due to the underperformance of the Zale brand and its impact on our consolidated result.”
Judging from headlines not pursued, they may have mucked with the books to get those results. Lots of people have resigned and there seems to be and SEC investigation
But my point—even with lots of DM testing, few tests generate 98 percent certainty.
What confidence level do DM companies really require before they change a control, and does it vary with the size of the company or with other factors?
DH to readers:
If anyone cares to jump into this debate, please do. Given yesterday’s special report in The Wall Street Journal, which contained an article, “Testing, Testing,” I get the sense that testing is the least understood aspect of marketing. Yet it is the most important.