In statistics, variation between the outcome measures of one study or between the results of similar studies is called ‘heterogeneity’. Multi-lab replications of a single original study present some interesting challenges: heterogeneity can be difficult to interpret.

The replication crisis was an important reason to initiate a grant for replication studies in The Netherlands (NWO, 2017-2019). How do the researchers who received such grants and do the actual replication work engage with this crisis in their own work? In this blog post, we will dive into these questions.

Time and budget restrictions as well as other funding criteria can shape how science is being done in practice. Here, we describe the impact of NWO funding specifications on “our” replication research projects. Funder stage: What kinds of replications should receive funding? In 2017, the Dutch science funding organisation NWO earmarked three million euros for … Continue reading How funding shapes research in practice →

In their article 'Self-correction in science: The diagnostic and integrative motives for replication', David Peterson and Aaron Panofsky (2021) distinguish two kinds of replication: diagnostic and integrative replication. In this blog post we discuss how well these categories apply to the studies in our sample, and we propose to distinguish a third kind of replication … Continue reading Kinds of replication →

How are the replication researchers we are studying doing in these corona times? In our first round of interviews, we noticed a strong impact from Corona restrictions and/or work overload in most of the NWO replication studies. Many of the studies are delayed and thus still running. To study “Replication in Action”, we first try … Continue reading Replication in Corona Times →

In this blog post we report first observations and impressions from our study following 24 recently funded replication studies in practice. NWO funding calls for replication studies: what were the characteristics? The aim of the three replication calls, according to the NWO website, was to encourage researchers to carry out replication research and thus to … Continue reading What characterizes replication studies? First impressions. →

In the last decade, concerns have been raised in several fields of research about the reproducibility of research findings. Our project, funded by the Dutch Research Council (NWO) aims to increase our understanding of the practical and epistemological issues at stake in replication studies. If a result is not reproducible in a replication study, does that mean that the original claim was false? And if not, how may we explain the failure to reproduce? Reproducibility of results by different researchers is generally considered an important touchstone of the validity of scientific claims, but there are a number of complexities associated with replication and reproducibility. We are going to study how these issues are dealt with in practice. In this post we describe how we will do this.