From the archive: S.G. Soal – A statistical master of deception Chris Scott The Skeptic

This article originally appeared in The Skeptic, Volume 2, Issue 2, from 1988.

The philosophical world was startled in 1944 by the publication from the head of the Cambridge philosophy department of an extraordinary paper: a 15-page discussion in the journal Philosophy on “The Experimental Establishment of Telepathic Precognition.”

Of course this was not the first time that a respected thinker had expressed belief in a fringe science. Newton himself was a devotee of alchemy, Alfred Russell Wallace was a firm believer in spiritualism, Sir William Crookes believed in the séance room materialisation of the dead. But this time it really looked different. To begin with, the author had a reputation as a supremely cautious man. Head of a faculty which included Bertrand Russell, Wittgenstein and Moore, his intellectual standing was beyond dispute.

Broad was not writing about his own experiments but about those of another academic, S.G. Soal, a mathematics lecturer at London University. Soal was a sceptic about psychical research. In the mid-1930s he had collected over 120,000 card guesses with no sign of above-chance results. He had become openly scornful of J.B. Rhine’s celebrated card-guessing experiments in America and Rhine’s claim to have demonstrated extrasensory perception (ESP). Yet it was these very results of Soal’s, apparently totally negative, that had led to the new findings. A colleague had urged Soal to look again at those discarded data and try the effect of scoring the guesses, not against the target card at which they were aimed, but against the cards just before and just after. As Soal later recounted, with much reluctance he made the check and found, to his surprise, two persons who had scored indisputably above the chance level on this new criterion. One of these subjects, Basil Shackleton, was then followed up by Soal in a new series of experiments and it was these that yielded the remarkable findings now signalled by Broad.

Hubert Pearce with the parapsychologist J. B. Rhine, experimenting with Zener cards. Public domain, via Wikimedia Commons

Soal’s experiments, unlike many others in the new field of “parapsychology”, appeared to be models of scientific rectitude. Extreme precautions were taken to eliminate normal means of communication, recording errors, cheating by the participants, and even fraud by the experimenters. The experiments continued over a total of 40 sessions. A large number of witnesses were brought in. Copies of the score sheets were made by hand at the end of each session, witnessed, and posted in the presence of witnesses to Professor Broad in Cambridge. The originals were stated to have been deposited in the archives of the Society for Psychical Research.

Shackleton scored consistently above chance throughout, but his success was with the card one ahead – the one to be looked at next. When the calling rate was doubled, his scoring shifted to the card two ahead. In some of the experiments the choice of each target card was made only at the moment of guessing, by taking a counter blind from a bag. This condition seemed to require precognition as the only explanation; it also ruled out any possibility of normal communication through some undetected channel. Finally, the statistical significance was overwhelming (over 10 to 1) so that chance as an explanation was clearly ruled out.

Over the years that followed, the Soal-Shackleton series was constantly cited by parapsychologists as a model; they saw in this work the conclusive evidence they had long awaited. Repeatedly it was said that only a person ignorant of the evidence could now rationally deny ESP. This theme was taken up by less committed but more influential writers, notably in England by C.E.M. Joad, the popular philosopher and broadcaster. Many open-minded people must have been led to see ESP as just another of the marvels, along with relativity and quantum theory, which modern science required them to believe in.

The full story of how these uniquely rigorous experiments were reduced to ruins is too complex to relate here. The process took thirty years and involved many independent researchers. We quote only the highlights.

In 1955, G.R. Price showed that several methods of fraud were possible if collusion were assumed between Soal and two other participants.

Between 1955 and 1960, C.E.M. Hansel developed a hypothesis of fraudulent card substitution by the agent (the person whose thoughts were supposedly being read by Shackleton). For this, Soal would need only one fraudulent collaborator in any given session. Over the whole series three different persons had operated as agent, selected by Soal. Some statistical features of the data seemed to lend some support to this hypothesis, though far from conclusively. Soal scornfully rejected Hansel’s reasoning: he successfully showed that the hypothesis could not account for all of the results.

In 1956, pressure from sceptics requesting the records for study obliged Soal to reveal that, contrary to the published report, the original records were not available because he had lost them. The handwritten duplicates that had been mailed to Broad were still in existence, held by the Society for Psychical Research.

In 1960, further pressure from sceptics obliged Soal to publish a more serious admission which had been known to some of them by hearsay. During the experiments one of the participants, Mrs G. Albert, had reported seeing Soal improperly “altering the figures” – specifically, changing ones into fours and fives. A full contemporary report had been kept by his collaborator, Mrs K.M. Goldney, but Soal had up to that point refused to allow publication, using the threat of a libel action.

In 1971, R.G. Medhurst reported a computerized attempt to identify the sequence of target digits that Soal had used. He was unable to trace the sequences in any relevant sources and concluded that Soal’s account of the way in which the targets were prepared must be incorrect.

In 1973, C. Scott and P. Haskell showed that there was strong statistical evidence in the experimental records supporting the allegation that Soal had “altered the figures”, changing ones into fours and fives in such as way as to turn misses into hits. The effects found were both highly significant statistically. They appeared in just three of the 40 sittings, and in particular in the one in which Albert had reported observing the manipulation, and they could account for the whole significant ESP-effect in these three sittings.

This damning evidence seems to have had little effect on the community of parapsychologists. Almost without exception those who commented rejected the fraud interpretation, despite the highly specific confirmation of Albert’s observation, and reaffirmed confidence in Soal.

The coup de grace came in 1978 when B. Markwick published the results of an astonishingly tenacious pursuit of the line opened earlier by Medhurst. She first showed that the target digits contained many runs of consecutive digits that were repetitions of runs used in earlier sessions, and that this could not have arisen by chance. This in itself was not particularly serious. However, she also showed that in many such cases the repetition was not exact but there were intrusions – digits inserted in the sequence – and that these intruded digits were nearly always hits. The implication of improper manipulation seems almost inescapable.

The mentalist Frederick Marion (back right) being tested at the National Laboratory of Psychical Research in a card experiment by the psychical researchers Samuel Soal (middle) and Harry Price (right). Source: Josef Kraus (Frederick Marion), Public domain, via Wikimedia Commons

After Markwick’s publication parapsychologists generally gave in and admitted that the evidence against Soal’s honesty was overwhelming. Many, however, continued to discuss the possibility that some of the data in the Soal-Shackleton series were still due to ESP. And most said that, despite their new-found belief in Soal’s duplicity, they had been right at the time to reject the Scott-Haskell evidence.

Clearly the views of parapsychologists are highly resistant to infection by unfavourable evidence. The total collapse of the strongest experimental series ever produced has not visibly diminished their belief in ESP. They simply cite different experiments now.

Recently, parapsychologists have argued that frauds have been identified in many branches of science without thereby discrediting the field of knowledge concerned: “why should parapsychology be treated any differently?”, they ask. They overlook a crucial fact: in ESP research, it is the very existence of ESP that is at stake. The only serious evidence for ESP comes from experiments, like Soal’s, in which guesses are aimed at random targets in conditions where normal communication is supposedly ruled out. But every one of these experiments has been less carefully controlled than the Soal-Shackleton series.

The Soal débacle is the clearest possible demonstration that believers in ESP are over-ready to accept defective evidence. To continue to accord the same value to the remaining experiments even after this demonstration implies an unshakeable faith impervious to counter-evidence.

The post From the archive: S.G. Soal – A statistical master of deception appeared first on The Skeptic.

From the archives in 1988, Chris Scott recounts Soal’s experiments, which suggested precognitive abilities, but turned out to be statistical fakery
The post From the archive: S.G. Soal – A statistical master of deception appeared first on The Skeptic.