The first part of the title of this article is based on the sub-title of one of the first articles on paranormal belief that I ever wrote, some three decades ago (French, 1992). Within parapsychology, the term sheep refers to believers in the paranormal, whereas sceptics are referred to as goats. This terminology is based upon a Biblical reference to Judgment Day (Matthew 2, 31-33), when apparently Christ will return and judge all nations, separating people, one from another “as a shepherd separates the sheep from the goats”.
The questions I addressed in that short overview were twofold. First, was there any evidence for the existence of cognitive biases that might sometimes lead people to misinterpret certain situations as involving paranormal phenomena when, in fact, they were better explained in non-paranormal terms? Secondly, if such biases do exist, are they stronger in believers than sceptics? I concluded that there was plenty of evidence in support of a positive answer to the first question and some suggestive evidence in support of a positive answer to the second.
Fifteen years later, in collaboration with Krissy Wilson, I published an updated review of the area in an edited volume, concluding that by then there was fairly consistent evidence that some cognitive biases were indeed associated with paranormal belief whereas the evidence was mixed with respect to other such biases. The relevant evidence was again discussed in my book, Anomalistic Psychology: Exploring Paranormal Belief & Experience, co-authored with Anna Stone.
I think I can be forgiven then for being somewhat irked to read the following sentences in a press release relating to a recent review of the same topic by a team at the University of Hertfordshire led by Charlotte Dean:
For several decades, researchers have examined potential links between cognitive functioning and belief in paranormal phenomena, such as psychokinesis, hauntings, and clairvoyance. However, about 30 years have passed since a non-systematic review of this literature was last conducted.
In case you were wondering, the non-systematic review being referred to here is not the one I wrote, it is an excellent review published by Harvey Irwin in 1993.
Needless to say, I was keen to read the paper by Dean and colleagues which had been published in the open access journal, PLOS ONE. The same false claim regarding 30 years passing since the previous review of the area is repeated in the paper itself. However, on the positive side, the paper does review a large number of relevant papers and it is indeed the first systematic review of the field. Neither our reviews nor that of Irwin were systematic reviews in the sense that we might currently understand that term. For example, we did not use pre-registered eligibility criteria to decide which papers to include and exclude as Dean and colleagues did. Having said that, our coverage of the field was pretty comprehensive nonetheless.
Dean and colleagues divided the 71 studies (total n = 20,993) that met their inclusion criteria into six categories: (a) perceptual and cognitive biases, (b) reasoning, (c) intelligence, critical thinking, and academic ability, (d) thinking style, (e) executive function and memory, and (f) other cognitive functions. The overall conclusion of their review was very much in line with that reached by ourselves in our previous reviews. There was fairly consistent evidence for stronger cognitive deficits for believers versus sceptics with respect to some cognitive biases and patchy or no evidence with respect to others. In the words of the authors:
The most consistent associations emerge for paranormal beliefs with increased intuitive thinking and confirmatory bias, and reduced conditional reasoning ability and perception of randomness.
The authors provide a valuable service by assessing the quality of the studies reviewed and it is reassuring to note that “quality was rated as good-to-strong for 75% of studies and appears to be improving across time”. However, some methodological weaknesses were noted, including: “the lack of preregistration, discussion of limitations, a-priori justification of sample size, assessment of non-respondents, and the failure to adjust for multiple testing.” It was also noted that, “Over 60% of studies have recruited undergraduates and 30% exclusively psychology undergraduates, which raises doubt about external validity.” All good points, well made.
I was, however, puzzled by something about this review article. Having worked in this area for several decades, I was surprised to see only three of my co-authored papers were included. A quick look at my CV revealed that this was less than half of the papers on my CV that appeared to me to be relevant to this topic. That got me wondering how many other relevant papers from other labs may not have been included. I have no reason to believe that inclusion of these other studies would have altered the overall conclusions drawn by the authors but, in the absence of an analysis including them, we simply cannot know for sure.
Why were these missing papers not included? The search strategy used by the authors looked reasonable at first sight, using “paranormal belief” AND <relevant cognitive function/bias> as the search terms in various electronic databases. But on reflection I realised that this may well miss many relevant papers. For example, Blagrove, French, and Jones (2006) refer to “belief in precognitive dreams” in the title, not “paranormal belief” – and yet this paper clearly is relevant to the topic. Wilson and French (2008-2009) refer to “misinformation effects” in the title but not “memory”, even though any memory researcher would know that this paper was dealing with a memory-related phenomenon. In the case of Brotherton and French (2014), paranormal belief was measured, analysed and reported but it was not the main focus of the study so may well be missed in any electronic literature search.
In the cases of Richards, Hellgren, and French (2014) and Wilson and French (2014), I am at a complete loss to understand why these papers were not included as both have “paranormal belief” and “memory” in the title. The lesson here appears to be that even systematic reviews may miss a lot of relevant research if the search terms used are not chosen very carefully.
One final very important point: Overall, the review offers support for the so-called “cognitive deficits hypothesis” of paranormal belief; that is the idea that believers in the paranormal tend to have stronger cognitive biases in at least some respects compared to sceptics. Personally, I believe that this hypothesis does indeed have considerable merit.
However, most experiments in this field have been designed, carried out, and reported by sceptical psychologists such as myself. Hardly any research has investigated the possibility of stronger cognitive biases on the part of sceptics in some situations. For example, would sceptics show any tendency to deny the evidence of their own eyes if, by deceptive means, an object appeared to move as a result of a (faked) psychokinetic effect? It may be reasonable to express doubt that it was really a psychokinetic effect but would some sceptics simply deny that the object had moved at all?
Perhaps such studies would reveal a somewhat different picture to that reported in this (and my own previous) reviews.
Reference
French, C. C. (1992) Factors underlying belief in the paranormal: Do sheep and goats think differently? The Psychologist, 5, 295-299.
The post When it comes to the paranormal, do ‘sheep’ and ‘goats’ think differently? It looks like they do appeared first on The Skeptic.
A new systematic review examining cognitive functions and paranormal belief raises a few questions as to which papers were included, and why some were exclude
The post When it comes to the paranormal, do ‘sheep’ and ‘goats’ think differently? It looks like they do appeared first on The Skeptic.