Researchers and journalists debate over pseudoscience allegations in consciousness theory

124 scientists signed a letter criticizing the media’s coverage of a consciousness theory. A Northeastern professor who signed the letter and the journalist who wrote The New York Times‘s coverage weigh on

Carl Zimmer, a journalist for The New York Times, walked into a Ballroom in Greenwich Village, where a New York University professor’s band was jamming out. The evening seemed a bit too exciting for a scientific conference. Consciousness science’s biggest names were all there, including Christof Koch, who had personally invited Zimmer. The evening culminated in researchers presenting results from a new study that pitted two prominent theories of consciousness head to head. It moved fast with flashy slides; Zimmer scribbled down notes and recorded everything.

“It was challenging because these theories for consciousness are all very complex, and the way that you test them experimentally itself is also very complex,” Zimmer said. “And they were ramming that stuff through pretty fast.”

In the young field of consciousness science, there has been little consensus, but now, some researchers are claiming one theory, Integrated Information Theory, or IIT, is accumulating significant evidence and support. Others fiercely disagree. One scientist told Zimmer the results of the study presented at the conference confirmed the predictions of IIT, and another said the results were mixed. One week later, The New York Times published Zimmer’s article with the headline “2 Leading Theories of Consciousness Square Off.”

Jorge Morales, a philosophy and psychology professor at Northeastern was also attending the conference and was even more off-put by the event. “The results were presented as an entertainment event, and that’s really not something that sits well with many people,” said Morales. “This is real science. The details are really tricky. It’s a very complex experiment.”

Just a few months later, Morales would sign a letter with over 120 fellow scientists calling IIT a pseudoscience. The signers are worried the public — and other scientists — have an incorrect view of the state of the field, partly due to the media for failing to highlight issues with the theory and present nuance. For the signers, this is a serious problem: IIT — which says computers, animals, fetuses, and many other complex systems have consciousness — has serious implications for many controversial political and social issues like AI, animal rights, and abortion.

The pseudoscience letter, published on a psychology preprint platform, spends its first two paragraphs criticizing journalists’ use of the words “dominant,” “leading,” and “well-established” to describe IIT, and for echoing exactly what IIT proponents have told them directly, absent of peer review or a critical lens.

Consciousness is an abstract, head-spinning concept; it’s what we call the fact that we experience the world. Scientists and philosophers have struggled to define the word for as long as it’s been around. Some have coalesced around consciousness being “something it is like to be” — stare at that phrase for a minute and it still hardly provides much clarity. This, unsurprisingly, has made consciousness an incredibly difficult thing to study.

The scientific study of consciousness is still in its infancy, and the number of theories attempting to explain consciousness is proliferating, as scientists in the field have generally been welcoming of out-there long-shot theories. It’s a common phenomenon in science: When the obvious approaches don’t yield much, throw everything and the kitchen sink at the problem and see if anything works. IIT is one of consciousness science’s many kitchen sinks.

IIT was born in 2004 at the University of Wisconsin–Madison as the brainchild of sleep researcher Giulio Tononi. His strategy was to essentially reinvent the wheel to better understand it: Started by defining its properties (it rolls), then determined what the physical object would have to look like for it to have these properties (it has to be round). He started by listing common traits of consciousness — like how experiences are complex, unique, and intrinsic to each person. Then, from this list of traits, he attempted to derive what a physical object would have to look like for it to have consciousness. 

He used math to define how fundamentally conscious that physical object is based on this theory. The resulting parameter is called “integrated information.” Examples of systems with highly integrated information include the human brain … and computers. The extremely consequential predictions that IIT makes, paired with the claims from leading scientists about how much evidence supports the theory, is what the letter signers say pushes the theory into the realm of pseudoscience.

There are plenty of ways to define pseudoscience — it’s as complex of a term as science itself. But a useful definition is through its social function: Pseudoscience aims to mimic science to gain credibility for claims. This can be done through bogus studies (as is the case with the anti-vax and flat Earth movements) or by simply pointing to legitimate science as evidence for something it doesn’t actually support (like when some tabloids reported that a NASA-funded study — in which particles traveled in the opposite direction than expected — had found evidence of a parallel universe where time flowed in reverse). This, the letter signers argue, is what happened with IIT.

***

I spoke with both Zimmer and Morales independently to understand where they agree, where they disagree, and — potentially — where things went wrong.

The following is a condensed and edited version of their answers.

Do you see any clear indicators that IIT is a pseudoscience?

Morales: The main problem is that their hypotheses are completely trivial. For example, they predicted that if someone was conscious of a visual stimulus, the visual cortex was going to be active. And that’s just the silliest of predictions — anyone can predict that. 

Then you add on top of that the fact that the organizers invited The New York Times and the magazines for Nature and Science. There were all these newspaper or magazine articles saying, “Oh my God, great event. Two theories were tested, and one of them makes more good points.” And it’s not just that event. It’s over a decade of hyping the theory in the media that really does not match the evidence that they have.

That’s where pseudoscience comes in. It’s the kind of thing that when you see it, you know it. At least for me and for many other philosophers, what makes something pseudoscience includes a psychological component. There has to be some intention of presenting something as scientific when it’s not. When you look at IIT, all your experiments have nothing to do with the mathematical formalisms of the theory. The evidence is so far and disconnected from reality that pretending that that’s evidence for your very complex, rather implausible metaphysical theory, that’s where the gap is.

When you sell it as a scientific theory — or even worse, the only theory that is seriously trying to tackle the consciousness problem — that intention of presenting something as science when there isn’t evidence to back it up, that mismatch is what really creates a problem and should make us worry about it. 

Zimmer: In science, there certainly are red flags that even non-scientists can recognize.  For example, if a particular study has some funky statistics where it seems like they’re trying to draw a remarkable conclusion from ambiguous evidence.

But I don’t see the papers being published on IIT, or other theories of consciousness, as having clear red flags. There certainly have been scientists who have taken issue with the evidence that has been put forward for these different theories, but to argue, “You’re showing me that evidence, but that doesn’t really prove what you claim it proves,” that’s a totally legitimate part of the scientific process.

What do you make of the letter calling IIT a pseudoscience?

Morales: I think the letter emerged as an exercise for trying to balance the narrative in the public sphere and also within academia outside of the consciousness field. If you are not in consciousness and you read these newspapers, you would think, “Oh yeah, there’s one theory of consciousness, and it’s the winner.” So, part of the idea of the letter was to say, “Hey, this media coverage is completely unhinged. It’s completely disproportionate to the evidence and the scientific rigor that we hope to get from any theory.” 

If you have like a pet theory of something — like how sound waves travel or how water moves around —  it might not matter too much. The problem with consciousness is that it’s something that a lot of people care about. We’re conscious. It matters. For example, abortion is one of the things mentioned in the letter, but think about end-of-life decisions, people with disorders of consciousness, people who are in a coma, children, animals. Any theory of consciousness has consequences for these — and the particular consequences aren’t necessarily good or bad. It’s just a way of saying this matters. Policy decisions might be made under the assumption that IIT is the leading theory of consciousness in the field, which is a distortion of how researchers within consciousness actually perceive things.

Zimmer: As a journalist, I have not reported on that letter. I’m certainly aware of it, but I have not interviewed the authors of that letter to get a deeper understanding of what their thinking is. I found the letter confusing because, on the one hand, it’s arguing that IIT is pseudoscience. But then it seems to be presenting the fact that it’s been covered in the news as evidence of it being pseudoscience. I don’t really see the evidence for their claims, and the letter itself just doesn’t hold together logically for me.

I’m really quite mystified. It’s not as if journalists are all part of the IIT club. In the article, I wrote that in this presentation at the conference, there were, in fact, two theories being presented, and I presented the other one hopefully just as clearly as IIT. I’ve written about other theories of consciousness in the past, and lots of journalists have written about other theories of consciousness in the past.

So, have there been published papers in which some scientists have challenged it? Yes, absolutely. And honestly, if you show me any prominent theory of consciousness, I can point you to some papers where people say that theory is wrong. I don’t recall ever seeing a published paper in a scientific journal laying out that it is pseudoscience. So I’m puzzled why — if this theory has been around for 15 years — this is the first time in just a pre-print posted online that a group of scientists started to argue that it’s pseudoscience. It seems strange to me why, if this is true, that claim has not made it into the same peer-reviewed scientific literature that IIT and many other theories of consciousness have.

What role can scientists play in creating a pseudoscience? What are their responsibilities to ensure a science doesn’t become a pseudoscience?

Morales: I think in general, scientists are heavily responsible for some of the misconceptions in the public medium. And there is research about this —  the main originators of exaggerated claims are scientists themselves, often through their universities’ press offices. When researchers talk to their press offices or journalists, they want to say, “Hey, we just discovered this amazing thing, and it’s the best thing ever,” and that’s never true. There are always caveats.

In the case of IIT, the researchers actually bypassed press releases. They are, themselves, doing the PR campaign. If you go to Scientific American, you see Christof Koch writing about IIT being the best thing since the dawn of time. For some reason, he has access to these publications. He can publish these things. 

I do think that there is a fine line between creating a narrative about the importance of a finding and outright just exaggerating it. We’re guilty of this not even just in the media, but in our scientific journals, too. People like to hype their findings because, I don’t know, they sound more interesting not just to journalists, but to other scientists.

I think narratives are important — for the general public and for scientists. We’re not so good at just processing raw data. But it’s very important to make sure that you keep yourself at bay and try to keep things interesting but truthful and proportional to the evidence. And it is often the case that we scientists don’t do that, in part because maybe if it sounds better, it will get into a better journal. If you get into a better journal, you might get more funding, more grants, more jobs. So the incentives are not great for keeping us humble. But I think that we can say, “Hey, this is great, but we didn’t control for these things,” or “This is great, but it’s limited to these species.” You need to be humble and accept that your research has limitations that have to be addressed in different studies or by different people.

What role can journalists play in creating a pseudoscience? What are their responsibilities to ensure a science doesn’t become a pseudoscience?

Zimmer: In journalism, we are not publishing scientific reviews. We are not publishing 10,000-word monographs on consciousness. I’m lucky if I can write 100 words about consciousness — and for an audience where I cannot take for granted that they all have PhDs in neuroscience. So, I think sometimes when scientists complain that some crucial nuance is lacking from reporting, sometimes that is confusing the role of journalism with the role of scientific papers.

We certainly have an obligation to vet the people that we’re writing about, and you do that by talking to other scientists and by trying to read widely. And I think that when you do that as a journalist, writing about IIT is a perfectly valid thing to do.

There are certainly cases of scientists who did great work, got tenure, and then their work went downhill. Look at history; that’s not a rare thing. It doesn’t matter if someone has a lot of accolades — there are tenured professors who would deny that HIV causes AIDS. That doesn’t make it true. Certainly, journalists have to be ready to recognize that they’ve made mistakes if they have highlighted science that turns out to be poorly done or wrong. I see it more as an ongoing process. There’s no quick easy “science legitimacy machine” that you can use. You have to be working on it all the time.

***

The letter, like the label “pseudoscience” itself, is a tool. It raises the alarm, hoping to teach scientists, journalists, and the public alike to tread with caution — to think more critically about IIT. The letter likely won’t kill IIT, but it might force the theory to better ground itself in disciplined scientific practice and send a signal to journalists that they should pay attention to the large number of scientists concerned about IIT.

Sometimes scientists push theories too far, and it requires a balancing force like the letter signed by Morales to keep science in check: The letter itself could just be an example of the checks and balances built into science working.