DOI: 10.1101/488353Dec 6, 2018Paper

Influence of multiple hypothesis testing on reproducibility in neuroimaging research

BioRxiv : the Preprint Server for Biology
Tuomas PuolivaliJ. Matias Palva

Abstract

Background: Reproducibility of research findings has been recently questioned in many fields of science, including psychology and neurosciences. One factor influencing reproducibility is the simultaneous testing of multiple hypotheses, which increases the number of false positive findings unless the p-values are carefully corrected. While this multiple testing problem is well known and has been studied for decades, it continues to be both a theoretical and practical problem. New Method: Here we assess the reproducibility of research involving multiple-testing corrected for family-wise error rate (FWER) or false discovery rate (FDR) by techniques based on random field theory (RFT), cluster-mass based permutation testing, adaptive FDR, and several classical methods. We also investigate the performance of these methods under two different models. Results: We found that permutation testing is the most powerful method among the considered approaches to multiple testing, and that grouping hypotheses based on prior knowledge can improve power. We also found that emphasizing primary and follow-up studies equally produced most reproducible outcomes. Comparison with Existing Method(s): We have extended the use of two-group and separate-c...Continue Reading

Related Concepts

Laboratory Procedures
Programming Languages
Psychology
Research
Computer Software
Random Drug Testing
Research Study
Statistical Analysis
Analysis
Study

Related Feeds

BioRxiv & MedRxiv Preprints

BioRxiv and MedRxiv are the preprint servers for biology and health sciences respectively, operated by Cold Spring Harbor Laboratory. Here are the latest preprint articles (which are not peer-reviewed) from BioRxiv and MedRxiv.