Despite wide distribution of the American Statistical Association’s (ASA’s) cautionary statement about misuse and abuse of p-values from null hypothesis significance testing (NHST) in publications and policy decisions in 2016, their (mis)use remains ubiquitous. The persisting need to modify inappropriate use of p-values is made evident by the special edition of The American Statistician (TAS) in 2019 led by the executive director of the ASA focusing on this topic. The edition highlights both existing and novel methods to combat egregious scientific conclusions using p-values, ranging from simple measures to accompany p-values to paradigm shifts in default scientific inference to embracing subjectivity—in place of claimed objectivity—as the solution. The question remains, why are these tools not used more, particularly by individuals already educated about them?
This talk will be constrained to circumstances requiring a binary decision, initial use of NHST, and reporting of p-values. It will be presented in three parts. First, select existing accompaniments to p-values to aid in their interpretation will be summarized, including a novel decision-theoretic method (joint work with Ken Rice, UW Biostatistics) along with recent examples of their implementation either in initial data analysis or critiques of reported results. Second, I’ll review why these changes are so difficult to implement, mostly summarizing the TAS 2019 special edition, adding some personal experience as a “boots on the ground” collaborative biostatistician. Finally, I’ll show how even if these tools are not directly used, lessons from these tools in the form of “rules of thumb” and altering inherent understanding of p-values may be applied immediately.
