Can courageous dialogue between scientists with competing theories eliminate confirmation bias?
Contrary to the idea that scientists are impartial data analysts, just like anyone else, they’re invested in being right.
Can courageous dialogue between scientists with competing theories eliminate confirmation bias? Learn about adversarial collaboration in the podcast featured below.
Learn about adversarial collaboration in the podcast featured above.
"Scientists are human too!
We think of ourselves as being very reasoned and very cold. Rational. But when it comes to our own theories, much of that falls by the way, and we are very biased. What I mean by that is: we look for confirmation...We go out of our way to invent all sorts of reasons why any data that's conflicting with our theory can’t be right. It wasn't collected in the right way, or there’s an exception, or this, that and the other." This is how Dr. Christof Koch, Meritorious Investigator, Allen Institute for Brain Science and chief scientific officer of the Tiny Blue Dot Foundation, describes the tendency that scientists have — just like any other human beings do — to defend what they’re passionate about and what they believe to be true.
Dr. Daniel Kahneman, Eugene Higgins Professor of Psychology, Emeritus, Princeton University explains, "Bias is really very insidious, and when you have a theory or a set of hypotheses about a phenomenon, and you design an experiment, you intuitively choose the experiment that is the most likely or highly likely to confirm your theory. I think the idealized model of science as being purely objective and so on is false, it’s misleading." This doesn't mean that scientists aren't conducting their work properly. "Even if we have done good data collection and good experiments, and we have done the analysis in a proper way, it's not impossible that we interpret the results in a biased way," says Dr. Lucia Melloni, Research Group Leader at the Max Planck Institute for Empirical Aesthetics.
Unfortunately, there is no systematic mechanism to prevent bias — it's part of human nature. "Science is not broken. But we have to admit that science is made out of scientists," quips Melloni.
The Challenges of Bias and Long-standing Industry Standards
Without a systematic mechanism preventing confirmation bias, scientists challenge their colleagues’ findings. That can be valuable and necessary, but being on the receiving end of critique isn't easy. Dr. Susan Fiske, recently retired from Princeton University as a professor of psychology and public affairs, says in its extreme, this kind of challenge can feel like "mob science." She shares that early in her career, when she was "more ambitious and cutthroat," she "would take somebody’s theory apart if it was competing" with hers. "It felt bad afterwards," she says. "You know, do I want to be somebody who goes around 'killing off other people's children?' I mean, that’s what you do in science, right? You say, 'Oh, your model's fine, but it's not really right. And here’s my data that destroys you.'"
Virginia Cooper, a Discovery Science advisor at Templeton World Charity Foundation, explains that there are also powerful incentives within the science industry for scientists not to trust each other, not to work together, and not to be open about process. She says, "If you want to get published in a high impact journal, you have to discover something. You have to prove something. You have to find the new patent. You have to do something that's going to get people interested. Unfortunately, in the way science is done now, from that top perspective, the incentives aren't to do good science. You're not incentivized to do all the sharing [that would happen with open science practices], you’re incentivized to get the paper published in that top impact journal."
A process called adversarial collaboration may offer an alternative to these harsh and inhibiting situations.
Listen to the above podcast to hear how this innovative process could transform science.
What is Adversarial Collaboration?
According to Melloni, "adversarial collaboration is a process by which two adversaries, or people who have opposing views, sit together, try to understand the theories of the others, try to come up with experiments, experimental designs, and predictions," that compare and test different hypotheses."
A neutral third party usually conducts the experiment and analyzes results. This is in contrast to how science is usually done, where the scientists who support a theory often also do their own experiments. "The key is that they agree ex ante that the methods are good to test their theories and that if the results go against their views, they will be willing to change their minds," Melloni explains, adding: "That's how it should work in practice. Now, the question is, are people able or not to change their minds? And who is supposed to change their minds?"
To participate in adversarial collaboration requires "a lot of intellectual humility," notes Cooper. This kind of collaboration offers a wider scope of understanding than traditional siloed practices. "I think adversarial collaboration adds a level of rigor to science because you have to be able to understand not just your theory, but the opposing theory as well," Cooper says. Melloni agrees. "The biggest lesson for me was understanding that is not necessary for the theory proponents to change their mind. It’s not necessary, not even wanted," she says. This is because science "happens through a process" and not "a destination." Adversarial collaboration helps us "understand that it's important to be able to change your mind over time."
From "Angry Science" to "Open Science"
Sharing outputs, codes, and "ways to make the scientific method more accessible to anybody that’s looking for it" allows the greater community to "do things like replicate your work or hold you accountable and see exactly what you were doing to get your results."
Open science "connects really well with adversarial collaboration" Cooper says. Adversarial collaborators have to be transparent in what they're doing. "If you are working with somebody you consider an adversary, you're working with a group of people who are not necessarily agreeing with you, but are answering your question from a different perspective."
The process has grown out of not only a desire for an alternative to what Kahneman calls "angry science," but because collaborating is actually a natural impulse. "Science operates in teams, and there's the excitement and the loyalty that you have when you're working with the same team of people," shares Fiske. "Then there's the competition between teams. And so it seemed to us to be more constructive, to be working together and leveraging each other's talent and energy and ability to run replications of each other."
Adversarial Collaboration, Team Science, and Flourishing
To make science "more robust," Melloni feels adversarial collaboration should be performed in all fields across the scientific community — period.
"It certainly could make things better for scientists because it illustrates a joint commitment to the scientific method." She further notes "team science is a collective of individuals with different competences that that come together to approach a problem and solve it together. The answer is in the collective intelligence — it is more than the sum of the parts."
Built upon the award-winning video series of the same name, Templeton World Charity Foundation’s “Stories of Impact” podcast features stories of new scientific research on human flourishing that translate discoveries into practical tools. Bringing a mix of curiosity, compassion, and creativity, journalist Richard Sergay and producer Tavia Gilbert shine a spotlight on the human impact at the heart of cutting-edge social and scientific research projects supported by TWCF
