A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000–2020)

Abstract

Meta-analysis is a powerful and important tool to synthesize the literature about a research topic. Like other kinds of research,meta-analyses must be reproducible to be compliant with the principles of the scientific method. Furthermore, reproducible meta-analyses can be easily updated with new data and reanalysed applying new and more refined analysis techniques. We attemptedto empirically assess the prevalence of transparency and reproducibility-related reporting practices in published meta-analysesfrom clinical psychology by examining a random sample of 100 meta-analyses. Our purpose was to identify the key points thatcould be improved, with the aim of providing some recommendations for carrying out reproducible meta-analyses. We conduct-ed a meta-review of meta-analyses of psychological interventions published between 2000 and 2020. We searched PubMed,PsycInfo and Web of Science databases. A structured coding form to assess transparency indicators was created based onprevious studies and existing meta-analysis guidelines. We found major issues concerning: completely reproducible searchprocedures report, specification of the exact method to compute effect sizes, choice of weighting factors and estimators, lackof availability of the raw statistics used to compute the effect size and of interoperability of available data, and practically totalabsence of analysis script code sharing. Based on our findings, we conclude with recommendations intended to improve thetransparency, openness, and reproducibility-related reporting practices of meta-analyses in clinical psychology and related areas.