What your mother failed to warn you about data: The Bradford-Hill criteria

In marketing, no one really dies if the end result does not meet expectation (i.e. the promo fails). However, when epidemiologists need to make a decision about intervening in the progress of a disease, the implications are life and death. For them, causality is no trivial matter.

In 1965, Sir Austin Bradford Hill produced a landmark paper containing several important lessons for interpreting data as an epidemiologist. These lessons apply equally to determining causality from data in a marketing environment.

1. Statistical significance should not be mistaken for evidence of a substantial association.
There may be a limited amount of attributes that determine your competitive standing in the market - for instance, if you're in the courier business, the speed with which you collect the parcel is of primary importance to the sender/purchaser (and not necessarily the speed with which you deliver it). Trying to differentiate yourself on that aspect (speed) requires operational expense and does not necessarily determine that the secretary will use you, just because you arrive 5 minutes sooner than the opposition. The task of getting her to call you every time (which is evidenced by keeping your waybills at the top of the pile), can sometimes appear as more subtle factors in the research, such as the ease of filling out the form or the frequency of interaction to maintain awareness.

2. Association does not prove causation
Just because consumers love a bargain, does not necessarily mean that the purchase in all cases is driven by price. Certain consumers (especially in complex product categories) may look to price as an indicator of quality or promise of value. Here the price point plays a greater role than relative price.
Retailers who upsell from the advertised item have known this for ages.

3. Precision should not be mistaken for validity.
When we take a number of measurements, such as asking people how many hours they watched TV last week, there will be some error in respondent replies - some will underestimate and others will overestimate. These errors usually cancel each other out. However, if the question the researcher asks is ambiguous, or follows another question that influences it, then we have precise, but bad data. Accuracy is not validity.

4. Evidence that there is a causal relationship is not sufficient to suggest action should be taken.
New Coke, introduced in 1985 to replace its flagship Coca-Cola, clearly illustrates this point. Public reaction was overwhelmingly negative, guaranteeing the new cola a place in the pantheon of marketing flops.
The research data (taste tests) showed that consumers clearly preferred the sweeter taste of Pepsi to Coke. Although there was a clear correlation, the sweetness of Coke was not the primary causal factor in the sale of Coke.

5. Uncertainty about whether there is a causal relationship is not sufficient to suggest action should not be taken.
Just because consumers have not complained about service, does not mean the service level can be excluded as a causative factor in the effect of patronage. Consumers have pressured lives and tend to spare their energy. When confronted with poor service, the strategy of the day is for them to cut their losses and not return, preferring to vent to a third party, such as a friend. There are numerous myths regarding the topic: a common one being that when a customer has a positive experience he tells three others, but when he has a negative experience he tells seven others. To test this hypothesis, John Goodman spoke to 1 700 Coca-Cola customers and found the results to be: told up to 5 people about a good experience and up to 10 people about a bad experience. It all depends on how emotionally involved the consumer is - for cars, the numbers are 8 and 16.

As marketers, we like to think that the causative factors we introduce for the desired effect is due to us, seeing ourselves as the main agents responsible for a brand's success. The truth is, we often are. So do what counts, and not just what you can count.

References
1. Carl V Phillips and Karen J Goodman. The missed lessons of Sir Austin Bradford Hill. Epidemiologic Perspectives & Innovations 2004, 1:3
2. David Boyle. The tyranny of numbers. Flamingo Books 2001.
3. Emanuel Rosen. The anatomy of buzz. HarperCollinsBusiness 2001.