Among researchers globally, there is increasing awareness of the benefits of preregistering a study or research project. Preregistration involves specifying a research plan – including hypotheses, methods, and analyses – in advance of a study and submitting it to a registry.
By separating hypothesis-generating (or exploratory) research from hypothesis-testing (or confirmatory) research, preregistration improves the quality and credibility of studies. By enhancing the transparency of researchers’ plans, methods, and any amendments as data becomes available, preregistration also assures scientists – and their colleagues – of the robustness of results and the degree to which the results can be trusted.
In addition, preregistration enables progress in science. An important part of open science is that it supports access to data and facilitates collaborative work across the research ecosystem.
Grantee and preregistration champion
Templeton World Charity Foundation grantee Prof. Liad Mudrik is a strong advocate for preregistration of research.
Now a co-director of an Accelerating Research on Consciousness adversarial collaboration grant, it was during Prof. Mudrik’s Ph.D. studies at Tel Aviv University that the importance of preregistration became apparent.
Studying cognitive neuroscience with a focus on the role of consciousness and attention in processing contextual violations, Prof. Mudrik’s research question considered whether integration processes could be performed between objects and scenes without awareness.
“We used a paradigm called continuous flash suppression, where a different image was presented to each eye. One was very colorful, flashing strongly and rapidly, so it dominates your perception and you don’t see the other image that’s presented to you,” explained Prof. Mudrik.
When the other image eventually emerged into participants’ awareness, they saw scenes where a person was performing an action with an object. The scenes were a mix of congruent images, such as a person drinking from a cup, and incongruent images, where a person was drinking from an object such as a plant or potato.
“What we found was that incongruent images emerged faster into participants’ awareness, so they broke the suppression faster. We concluded that when one of the image types emerges faster, it might mean that unconsciously the scene and the object have been integrated and their relations have been deciphered – and that’s why the incongruent images emerge faster,” said Prof. Mudrik.
Shortly after publication of her results, Prof. Mudrik was contacted by Prof. Christof Koch from the California Institute of Technology, USA, who together with Prof. Giulio Tononi suggested a similar experiment as a test for consciousness. The pair predicted a different result to Prof. Mudrik’s findings.
With their subsequent discussion leading to Prof. Mudrik commencing postdoctoral studies with Prof. Koch, she was soon contacted by another researcher who had tried to replicate her findings and could not.
“I was a reviewer on the paper and found no reason to think that their replication attempt was problematic, so I had to test if the finding might have indeed been a fluke. In my own lab, I tried to replicate a similar finding and also couldn’t. This taught me many things,” explained Prof. Mudrik.
“I no longer publish papers without replicating the effect myself in my own lab. I also don’t publish papers based on one study.
“When I’m conducting an exploratory study, I proceed with preregistration. I describe my methods and analysis detailing exactly what I want to do, and that provides some defense against different types of issues that in retrospect might have prevented the failure to replicate, while making the entire process transparent and open.”
Preregistration enables, rather than limits, curiosity
Since 2019, Prof. Mudrik has been awarded Templeton World Charity Foundation grants for three projects. It is her current work exploring two prominent theories of consciousness with Prof. Lucia Melloni and Prof. Michael Pitts where preregistration has been particularly important.
The project, An Adversarial Collaboration to Test Contradictory Predictions of Global Neuronal Workspace and Integrated Information Theory, explores which of the two theories has higher explanatory power when tested directly against each other.
“One of the consortium members said recently that we treat preregistration like the Bible in the sense that we go back to it and try to interpret what we meant when we wrote it, and make sure our analyses are directly reflecting that,” said Prof. Mudrik.
“For this specific project, the preregistration has been extremely valuable because it allows us to track the way our ideas are perfected and change over time, and some nuances in the way we think about things the further we are into the project, and understand its intricacies.
“That’s natural, it’s how human beings behave and think: once we get more information we can revise our prior, create a posterior – which becomes our new prior – and we just update our beliefs all the time. The important thing here is that the entire process is documented, so one could track the different stages of the project, and how the ideas evolved.”
Why and how to preregister research
A common misconception is that preregistration will “kill creativity in science,” with Prof. Mudrik saying many researchers are concerned that it will limit them to one specific analysis scheme.
“Think of preregistration as a declaration of intent of what you wanted to do before you saw the data – then if it reveals something new or interesting, that’s great, go pursue it. You just have to be transparent about the fact that this is an exploratory analysis,” explained Prof. Mudrik.
“If, on the other hand, you are only presenting the one analysis that has worked out of many different types that were conducted beforehand, then the information you’re giving is partial and could even be misleading.”
Prof. Mudrik added that preregistration is a useful way for researchers to document their initial thinking.
“You write based on your existing intent and knowledge, but you also have to think about your future self who will read what you wrote several months or years later and may not understand what you meant. This is why you want to be as detailed and explicit as possible, hereby also restricting your own degrees of freedom – yet not for explorations!” said Prof. Mudrik.
As an associate editor of international journal Consciousness and Cognition, Prof. Mudrik says she considers it a merit of the research if it has been preregistered.
While there are several prominent research registries, Templeton World Charity Foundation recommends the Center for Open Science’s OSF Registries.
The center’s website has a range of resources for researchers preregistering a project, including instructions, tips, and templates.
“It’s exciting to live in a time where the rules of the game are being changed. We’re able to think not just about the sort of questions we want to ask as scientists, but also think meta-scientifically about the practices by which we pursue and study those questions,” said Prof. Mudrik.
“For me as a researcher, it’s exciting to be part of such a project and try to make changes both within my lab and hopefully outside of it.”
This blog was inspired by a panel discussion with Prof. Mudrik as part of Templeton World Charity Foundation’s Open Science webinar series. Hear more from Prof. Mudrik on the benefits of preregistration.
