WHSC Library Seminar Series on Rigor and Reproducibility presents: Significance testing is the reason that scientific results have poor reproducibility.
Date/Time: Thursday, April 21st at 2:00pm
Presenter: Timothy Lash, DSc, MPH, Rollins Professor and Chair of Epidemiology, Rollins School of Public Health.
In the last years, some in the scientific community have raised alarms about a perceived lack of reproducibility of scientific results. In reaction, guidelines for journals have been promulgated and grant applicants have been asked to address the rigor and reproducibility of their proposed projects. Neither solution addresses a primary culprit, which is the culture of null hypothesis significance testing that dominates statistical analysis and inference. In an innovative research enterprise, selection of results for further evaluation based on null hypothesis significance testing is doomed to yield a low proportion of reproducible results and a high proportion of effects that are initially overestimated. In addition, the culture of null hypothesis significance testing discourages quantitative adjustments to account for systematic errors and quantitative incorporation of prior information. These strategies would otherwise improve reproducibility and have not been widely inculcated in the literature on this topic. Without discarding the culture of null hypothesis significance testing and implementing these alternative methods for statistical analysis and inference, all other strategies for improving reproducibility will yield marginal gains at best.
An overview of previous seminars and recordings can be found at https://guides.libraries.emory.edu/rigor-rep.