An estimate of the science-wise false discovery rate and application to the top medical literature

Biostatistics
Leah R Jager, Jeffrey T Leek

Abstract

The accuracy of published medical research is critical for scientists, physicians and patients who rely on these results. However, the fundamental belief in the medical literature was called into serious question by a paper suggesting that most published medical research is false. Here we adapt estimation methods from the genomics community to the problem of estimating the rate of false discoveries in the medical literature using reported $P$-values as the data. We then collect $P$-values from the abstracts of all 77 430 papers published in The Lancet, The Journal of the American Medical Association, The New England Journal of Medicine, The British Medical Journal, and The American Journal of Epidemiology between 2000 and 2010. Among these papers, we found 5322 reported $P$-values. We estimate that the overall rate of false discoveries among reported results is 14% (s.d. 1%), contrary to previous claims. We also found that there is no a significant increase in the estimated rate of reported false discovery results over time (0.5% more false positives (FP) per year, $P = 0.18$) or with respect to journal submissions (0.5% more FP per 100 submissions, $P = 0.12$). Statistical analysis must allow for false discoveries in order to ...Continue Reading

References

Apr 13, 1991·Lancet·P J EasterbrookD R Matthews
Mar 15, 1986·British Medical Journal·M J Gardner, D G Altman
Apr 5, 1997·BMJ : British Medical Journal·R BhopalH Rodgers
Jun 26, 1999·Annals of Internal Medicine·S N Goodman
May 8, 2001·Journal of Computational Biology : a Journal of Computational Molecular Cell Biology·M A NewtonK W Tsui
Mar 22, 2002·NeuroImage·Christopher R GenoveseThomas Nichols
Jun 1, 2002·JAMA : the Journal of the American Medical Association·Douglas G Altman
Jul 12, 2002·Genetic Epidemiology·Bradley Efron, Robert Tibshirani
Jul 29, 2003·Proceedings of the National Academy of Sciences of the United States of America·John D Storey, Robert Tibshirani
Oct 16, 2004·BMJ : British Medical Journal·Erik von Elm, Matthias Egger
Apr 14, 2005·World Journal of Surgery·Douglas G Altman
Aug 3, 2005·PLoS Medicine·John P A Ioannidis
Dec 22, 2005·Nature Reviews. Genetics·David B AllisonMahyar Sabripour
Apr 26, 2007·PLoS Medicine·Steven Goodman, Sander Greenland
Nov 27, 2008·Proceedings of the National Academy of Sciences of the United States of America·Jeffrey T Leek, John D Storey

❮ Previous
Next ❯

Citations

Apr 15, 2016·PeerJ·Chris H J HartgerinkMarcel A L M van Assen
Feb 28, 2015·Science·Jeffery T Leek, Roger D Peng
Feb 12, 2015·Proceedings of the National Academy of Sciences of the United States of America·Jeffrey T Leek, Roger D Peng
Sep 7, 2014·British Journal of Anaesthesia·M S Avidan, T S Wildes
Jul 8, 2016·Journal of Medical Internet Research·Sven Van PouckeMilan Vukicevic
Jul 23, 2016·Biostatistics·Yoav Benjamini, Rami Cohen
Nov 24, 2016·Biological Reviews of the Cambridge Philosophical Society·Wolfgang ForstmeierTimothy H Parker
Jul 31, 2016·Perspectives on Psychological Science : a Journal of the Association for Psychological Science·Prasad PatilJeffrey T Leek
Oct 4, 2016·Perspectives on Psychological Science : a Journal of the Association for Psychological Science·Jeff Miller, Rolf Ulrich
Mar 15, 2015·PLoS Biology·Megan L HeadMichael D Jennions
May 8, 2016·Behavioural Brain Research·Kendra E Hargis, Eric M Blalock
Mar 13, 2016·Statistics in Medicine·Susan Gruber, Eric Tchetgen Tchetgen
May 15, 2015·Clinical and Experimental Pharmacology & Physiology·Marco Marozzi
Apr 13, 2017·Journal of Clinical Epidemiology·Thomas V Perneger, Christophe Combescure
Oct 7, 2017·Research Synthesis Methods·George Karabatsos
Dec 9, 2017·Clinical & Translational Oncology : Official Publication of the Federation of Spanish Oncology Societies and of the National Cancer Institute of Mexico·A Carmona-BayonasL P Valiente
Jan 3, 2019·PloS One·Jeff Miller, Rolf Ulrich
Apr 13, 2019·PloS One·Robbie C M van AertMarcel A L M van Assen
Feb 23, 2019·Psychological Science·Anton Olsson-CollentineChris H J Hartgerink
Sep 11, 2019·HNO·C Kiese-Himmel, S K Plontke
Aug 31, 2019·Archives of Sexual Behavior·Justin P DubéNatalie O Rosen
Sep 16, 2018·BMC Bioinformatics·Matthew M ParksCharles E Lawrence
Jul 21, 2020·Physiological Genomics·Oliver HinesYalda Jamshidi
Nov 16, 2019·Pharmaceutical Statistics·Daniele De Martini
Mar 14, 2018·Proceedings of the National Academy of Sciences of the United States of America·Daniele Fanelli
Nov 28, 2018·Nature Materials·Andrew E Teschendorff
Feb 18, 2016·PloS One·Stephan B Bruns, John P A Ioannidis
Nov 27, 2015·BMC Research Notes·Bastiaan GinselIan Harris
May 28, 2016·Advances in Physiology Education·Douglas Curran-Everett
Sep 30, 2017·Journal of Evaluation in Clinical Practice·Per Lytsy
Jan 15, 2021·Biomaterials Science·Farooq AhmadTahir Muhmood
Jan 21, 2021·Annual Review of Public Health·Roger D Peng, Stephanie C Hicks
May 3, 2018·Trends in Genetics : TIG·Urko M MarigortaArcadi Navarro
Sep 22, 2019·Journal of Nuclear Medicine : Official Publication, Society of Nuclear Medicine·Irène Buvat, Fanny Orlhac

❮ Previous
Next ❯

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.