Reproducibility of preclinical animal research improves with heterogeneity of study samples

PLoS Biology
Bernhard VoelklHanno Würbel

Abstract

Single-laboratory studies conducted under highly standardized conditions are the gold standard in preclinical animal research. Using simulations based on 440 preclinical studies across 13 different interventions in animal models of stroke, myocardial infarction, and breast cancer, we compared the accuracy of effect size estimates between single-laboratory and multi-laboratory study designs. Single-laboratory studies generally failed to predict effect size accurately, and larger sample sizes rendered effect size estimates even less accurate. By contrast, multi-laboratory designs including as few as 2 to 4 laboratories increased coverage probability by up to 42 percentage points without a need for larger sample sizes. These findings demonstrate that within-study standardization is a major cause of poor reproducibility. More representative study samples are required to improve the external validity and reproducibility of preclinical animal research and to prevent wasting animals and resources for inconclusive research.

References

Nov 4, 2000·Nature Genetics·H Würbel
Jul 17, 2001·Annals of Internal Medicine·A R LocalioS E Kimmel
Oct 31, 2002·Nature Neuroscience·Elissa J CheslerJeffrey S Mogil
May 28, 2003·Journal of Clinical Epidemiology·Mirjam MoerbeekMartijn P F Berger
Jul 31, 2003·Genes, Brain, and Behavior·H Würbel
Feb 26, 2004·Statistics in Medicine·UNKNOWN Committee for Proprietary Medicinal Products (CPMP)
May 5, 2007·Brain : a Journal of Neurology·H Bart van der WorpMalcolm R Macleod
Sep 4, 2007·Trends in Neurosciences·Emily SenaMalcolm Macleod
Jun 23, 2009·BMC Medical Research Methodology·Emilie Vierron, Bruno Giraudeau
Mar 3, 2010·Nature Methods·S Helene RichterHanno Würbel
Mar 8, 2011·Parkinsonism & Related Disorders·Evelien D M RookeMalcolm R Macleod
Sep 6, 2011·Nature Reviews. Drug Discovery·Florian PrinzKhusru Asadullah
Mar 31, 2012·Nature·C Glenn Begley, Lee M Ellis
Oct 17, 2012·Statistical Methods in Medical Research·Joanna IntHoutGeorge F Borm
Feb 19, 2013·Lancet Neurology·Tzu-Ching Wu, James C Grotta
Apr 18, 2013·BMC Medical Research Methodology·Brennan C Kahan, Tim P Morris
Nov 13, 2013·Proceedings of the National Academy of Sciences of the United States of America·Valen E Johnson
Jun 10, 2015·PLoS Biology·Leonard P FreedmanTimothy S Simcoe
Oct 3, 2015·PLoS Biology·John P A IoannidisSteven N Goodman
May 24, 2016·Trends in Pharmacological Sciences·Bernhard Voelkl, Hanno Würbel
Jun 3, 2016·Science Translational Medicine·Steven N GoodmanJohn P A Ioannidis
Jul 31, 2016·Perspectives on Psychological Science : a Journal of the Association for Psychological Science·Martin S HaggerMaria Zwienenberg
Oct 28, 2016·Perspectives on Psychological Science : a Journal of the Association for Psychological Science·Alberto AcostaRolf A Zwaan
Nov 24, 2016·Biological Reviews of the Cambridge Philosophical Society·Wolfgang ForstmeierTimothy H Parker
Feb 12, 2017·Science·Eric Loken, Andrew Gelman
Apr 28, 2017·Nature Methods·Neri KafkafiYoav Benjamini
Jan 1, 2018·Nature Human Behaviour·Valentin Amrhein, Sander Greenland
Jan 10, 2017·Nature Human Behaviour·Marcus R MunafòJohn P A Ioannidis

❮ Previous
Next ❯

Citations

Sep 12, 2018·Journal of Cerebral Blood Flow and Metabolism : Official Journal of the International Society of Cerebral Blood Flow and Metabolism·Tuuli M HietamiesTerence J Quinn
Sep 9, 2018·Disease Models & Mechanisms·John P Sundberg, Paul N Schofield
Apr 3, 2019·Database : the Journal of Biological Databases and Curation·Yiqing ZhaoMary Shimoyama
Apr 17, 2019·Animals : an Open Access Journal From MDPI·Simone KumstelDietmar Zechner
Feb 5, 2019·GigaScience·Mark N WassMartin Michaelis
Apr 27, 2018·EMBO Reports·Thiago Fa França, José Maria Monserrat
Nov 20, 2019·Animals : an Open Access Journal From MDPI·Adrian J Smith, Elliot Lilley
Dec 18, 2019·Connective Tissue Research·Solaiman TarafderChang H Lee
Jan 25, 2020·Pain·Carole A FedericoJonathan Kimmelman
Oct 16, 2019·PLoS Biology·Bettina BertGilbert Schönfelder
Feb 6, 2019·ELife·Olavo B AmaralClarissa Fd Carneiro
Jun 5, 2019·Scientific Reports·Carina BoddenS Helene Richter
May 10, 2020·Cardiovascular Research·Neil MacRitchie, Pasquale Maffia
Jun 23, 2020·Clinical Pharmacology and Therapeutics·James W T YatesPaolo Vicini
Apr 24, 2019·Lab Animal·Johanna Åhlgren, Vootele Voikar
Jan 18, 2020·The International Journal of Cardiovascular Imaging·A FaragliA Alogna
Jul 17, 2020·Scientific Reports·Jeremy D BailooHanno Würbel
Jun 4, 2020·Nature Reviews. Neuroscience·Bernhard VoelklHanno Würbel
Jan 11, 2020·Frontiers in Pharmacology·Cordula Hirsch, Stefan Schildknecht
Jul 15, 2020·PLoS Biology·Nathalie Percie du SertHanno Würbel
Aug 21, 2020·Gates Open Research·Jade Benjamin-ChungBenjamin F Arnold
Nov 9, 2018·Journal of Translational Medicine·Pandora Pound, Merel Ritskes-Hoitinga
Jun 17, 2020·Pharmacological Reviews·Wolfgang LöscherAnnamaria Vezzani
Mar 20, 2019·Frontiers in Behavioral Neuroscience·Stina LundbergErika Roman
Jul 3, 2020·Nature Reviews. Drug Discovery·Kimberly Scearce-LevieJoseph W Lewcock
Aug 23, 2020·Nature Reviews. Neuroscience·Hanno WürbelTom Van de Casteele
Aug 23, 2020·Nature Reviews. Neuroscience·S Helene Richter, Vanessa von Kortzfleisch
May 12, 2020·Frontiers in Neuroscience·Sophie Helene Richter
Oct 8, 2020·Scientific Reports·Vanessa Tabea von KortzfleischS Helene Richter
Oct 24, 2020·ILAR Journal·Vanessa K LeeMichael J Huerkamp
Jan 13, 2021·ELife·Natascha Ingrid DrudeUlf Toelch
Jan 23, 2021·Journal of Animal Physiology and Animal Nutrition·Nadine PaßlackJürgen Zentek
Oct 22, 2019·The Lancet Infectious Diseases·Ignacio RubioJesús F Bermejo-Martín

❮ Previous
Next ❯

Software Mentioned

CAMARADES
Mathematica
R package metafor1

Related Concepts

Related Feeds

Cachexia & Brown Fat

Cachexia is a condition associated with progressive weight loss due to severe illness. In cancer patients, it is proposed to occur as a result of tumor-induced energy wasting. Several proteins have been implicated in browning and depletion of white adipose tissue. Here is the latest research on cachexia and brown fat.

Cardiac Cachexia

Cardiac cachexia is a syndrome associated with the progressive loss of muscle and fat mass. It most commonly affects patients with heart failure and can significantly decrease the quality of life and survival in these patients. Here is the latest research on cardiac cachexia.