There’s lies, damn lies and statistics

graph of spurious correlation

Clear correlations “proven” statistically.

There’s a reason I stick with my Twitter feed. It brings up articles that I otherwise would have missed. Take these two.


It’s an interesting re-analysis of a climate change article by Lewandowsky, Gignac, & Oberauer, (2013) presented by Dixon and Jones (2015), who concluded: “respondents convinced of anthropogenic climate change and respondents skeptical about such change were less likely to accept conspiracy theories than were those who were less decided about climate change.” This was in response to Lewandosky et. al. who had shown a robust linear relationship between climate change rejection and conspiracy theory ideation.

The rebuttal by Lewandowsky, Gignac & Oberauer (2015) seems pretty compelling. The essence of the argument is methodological. When do you choose the statistical model to use and believe? The main point that Lewandowsky make is:

“Dixon and Jones’s core argument is that the relationship between the two variables of interest, conspiracist ideation (CY) and acceptance of climate change (CLIM), is nonlinear, and that the models reported for both surveys were misspecified. To reach their conclusion, Dixon and Jones first make three questionable data-analytic choices to cast doubt on and attenuate the linear effects reported, before they purport that there is nonlinear relationship after reversing the role of the variables of interest in the statistical model for the panel survey. No statistical or theoretical justification for that reversal is provided, and none exists.”

So you can choose a different model but if you do, you better have a compelling reason for doing so. Dixon and Jones didn’t.

Hence, Lewandowsky et. al, conclude:

“In summary, Dixon and Jones’s analysis has no bearing on the results we reported for either survey because it reaches its main conclusion only by reversing the role of criterion and predictor without any theoretical justification. The only statistical justification offered for that reversal (“with nonlinear models, it is important to explore relationships in both directions”) demonstrably does not apply. Without that reversal, Dixon and Jones’s criticism involving nonlinear relationships is moot because none are present.”

The main point was elegantly stated a bit earlier in the article.

“Any correlation matrix can be fit equally well by more than one model. This issue of equivalent models has been discussed repeatedly (e.g., Raykov & Marcoulides, 2001; Tomarken & Waller, 2005). The consensus solution is to limit the models under consideration to those that have a meaningful theoretical interpretation (MacCallum, Wegener, Uchino, & Fabrigar, 1993). Alternative models should reflect alternative theoretically motivated hypotheses, any mention of which is conspicuously lacking in Dixon and Jones’s Commentary.”

There are lies, damn lies and statistics. If you are going to use statistics wisely you better have a good theoretical model on which to base you proposed analysis. In the absence of that any conclusion drawn is suspect.

J.B.S. Haldane, “[T]he Universe is not only queerer than we suppose, but queerer than we can suppose.”

Sorry that at least the second response article of these two is proprietary rather than open access. If you have access to a library that carries Psychological Science here are the relevant citations:

The open one with a CC BY NC is:

Ruth M. Dixon and Jonathan A. Jones. Conspiracist Ideation as a Predictor of Climate-Science Rejection: An Alternative Analysis
Psychological Science May 2015 26: 664-666, first published on March 26, 2015 doi:10.1177/0956797614566469

and, this one in rejoinder is behind a pay wall. 😦

Stephan Lewandowsky, Gilles E. Gignac, and Klaus Oberauer. The Robust Relationship Between Conspiracism and Denial of (Climate) Science. Psychological Science May 2015 26: 667-670, first published on March 26, 2015 doi:10.1177/0956797614568432

About longpd

I'm a senior scholar at Georgetown University, in CNDLS, a technologist and lapsed evolutionary biologist with an incandescent passion for new modes of seeing and learning.
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s