Could AI generated summaries of scientific consensus help tackle disinformation? Alfie Hoar The Skeptic

To anyone who campaigns against the influence of scientific disinformation – or frankly anyone who looks up a politically topical scientific field on social media – the emphasising of doubt and uncertainty by those who wish to undermine ‘mainstream’ science will be well recognised. It’s a very successful technique, and one that plays a central role in the Tobacco Industry method, which has been used by espousers of scientific disinformation for decades.

Emphasising doubt in science is a difficult thing to counter, since technically there is always doubt. It’s the nature of the scientific reasoning that ideas cannot be proved but rather supported or confirmed.

A common technique that is used to combat such rhetoric is to emphasise the mainstream thought within the scientific literature, either referencing widely accepted science or asking established scientists to refute the disinformation. However, this is more often than not spun into an idea of ‘debate in science’ – it’s happened to me! Again, this is frustrating because it’s widely known that there is debate in science and so it’s difficult to convey that this description isn’t an accurate description of the state of the science.

One of the major obstacles in these cases is being able to convey that the ‘debate’ here is not one between equally valid descriptions of the science – one take on the state of the science is a far more accurate description of the other. This is where the work of Consensus comes into play.

Consensus is a company based in the US which has the aim of democratising scientific knowledge. The company have developed a programme which is able to analyse a set of millions of scientific papers and generate a consensus response to a question posed to it. Thus it is able to identify the scientific consensus on a topic which is emergent from the scientific literature.

This idea of emergence is important, I believe, philosophically. A point which is often raised to dismiss the idea of a scientific consensus, normally around climate science, is that the very idea of consensus is unscientific. This often stems from the idea that uncertainty is inherent in science, or a Popperian idea that scientists should be constantly questioning the status quo.

However, this is a misunderstanding of what we’re talking about when we discuss a consensus in science. Any consensus we’re interested in here is about the convergence of the relevant literature around a certain position. This is what the programme Consensus has developed can provide.

It’s important to note at this stage that it’s early days for Consensus, with the programme only launching in early 2022. It cannot generate a consensus position on all questions yet, and sometimes the results generated when asking a question aren’t entirely representative of the scientific literature, but the programme is being continuously updated to improve the results generated.

Even at this stage, the usefulness of Consensus in tackling scientific disinformation can be easily seen. Take the recent press release from 2022 Rusty Razor winner Global Warming Policy Foundation, titled “Coral Reefs Are Not Declining”, which is to promote their new paper on the state of coral reefs globally. If one takes this press release title, and asks Consensus “are coral reefs declining?”, the result is a response showing that the scientific literature disagrees with the GWPF paper’s conclusion.

Similarly, we can take a topic which climate change denialists love to bring up: polar bears. The GWPF have run a series of papers arguing the polar bear numbers aren’t declining. If one asks Consensus “are polar bear numbers declining” one gets a response that, whilst not disagreeing with the GWPF’s stance as strongly as with regards to coral reefs, nonetheless shows that the scientific literature broadly holds that polar bear numbers are likely declining.

Again, I’m not claiming we should take these two results as a highly accurate description of the scientific literature at this stage. This test is simply designed to demonstrate the utility of such a programme in tackling scientific disinformation.

The examples above might seem like small fish to fry, but such disinformation has real-world effects. The GWPF, for example, regularly supply testimonies to UK Parliamentary committees, and so have a direct effect of climate legislation. It’s widely noted that politicians often don’t have the time or expertise to evaluate the accuracy of scientific advice given to them, often with such decisions relying on the credentials of the individual. With a disinformation network containing many scientists (often commenting on areas outside of their expertise, something that can be difficult for politicians to realise in a time-pressured environment) it’s understandable why anti-science promoters are often able to wrongly paint the situation as a debate amongst scientists, where the science unsettled.

A matured version of the programme from Consensus would go a long way in resolving this difficulty. Mainstream scientists could quickly demonstrate that their position is more reflective of the scientific literature to policy makers, making it far more difficult to construe this debate narrative and delay action.

There are still limitations to this, however. Firstly, climate science is a reasonably mature field with a consensus on topics going back decades. If we look back to the Covid-19 pandemic, research on some topics was very novel. Some initial ideas that seemed widely accepted changed, mainly due to how quickly the research was being done. The difference between these state of affairs needs to be recognised and qualified.

Secondly, it seems somewhat inevitable that the focus of doubt will shift away from the science and towards the algorithms being used. Promoters of scientific disinformation are likely to point to algorithmic bias if the programme is used to dismiss their position. Therefore, openness about how the programme works will be necessary if it’s going to be used to help inform policy makers. It also seems clear that this means that such a programme could never replace scientists advising policy makers. There will always be a place of scientists around the table, it’s just that a programme like Consensus would be a very useful tool for them.

I want to end this piece with a disclaimer. I don’t want this to appear as if I’m arguing that AI generated scientific consensuses will be the end of scientific disinformation. That’s far too optimistic. The networks that generate disinformation will evolve and will likely always exist. However, this work does provide some hope for those who wish to see such networks’ influence diminished, and our political system’s ability to respond to new scientific research improved.

The post Could AI generated summaries of scientific consensus help tackle disinformation? appeared first on The Skeptic.

Summaries of the scientific consensus, generated quickly by artificial intelligence might offer new hope in beating a key part of the ‘Tobacco Playbook’ on science denialism
The post Could AI generated summaries of scientific consensus help tackle disinformation? appeared first on The Skeptic.