Skip to main content

Null Regions: A Unified Conceptual Framework for Statistical Inference

Yuichi Shoda Headshot

Yuichi Shoda, Professor of Psychology, University of Washington

Abstract: 

Traditional null hypothesis significance testing (NHST) can be counterproductive to scientific progress. For example, increasing the precision of a study (e.g., by increasing the sample size) makes it easier to find statistical significance, with the result that any theory predicting a non-zero effect is considered supported—even if it explains only a small portion of the phenomenon of interest. This is not the case when using the "null regions framework" (https://doi.org/10.1098/rsos.221328) if the observed value falls within the “null region”—the range of population values that researchers aim to rule out (e.g., effects or associations that are too small to matter, even if not exactly zero). This framework provides tests to address questions such as: “Is the effect strong enough to matter?” and “Is the effect close enough to the predicted value to suggest that the theory that predicted it should be retained?” These tests can be conducted without computational tools, as long as confidence intervals for the statistic of interest are available. Exact p-values can be obtained as long as one has access to a function (e.g., in R) that computes confidence intervals for the statistic. The key role of confidence intervals in this approach encourages researchers to focus on estimating effects rather than adhering to the binary "significant" vs. "not significant" dichotomy. In addition, the null regions framework offers a more meaningful definition of successful replication and empirical confirmation of the predictions made in registered reports.

 

Shoda was born and grew up in Japan. He studied physics at Hokkaido University in Sapporo. After attending the University of California, Santa Cruz, he started graduate school in psychology at Stanford, and finished at Columbia University with a PhD degree in psychology in 1990. He joined the University of Washington in 1996.

Throughout his career, Shoda’s work focused on improving the methods and practice of behavioral science. He co-developed the “null regions” framework for statistical inference to solve the persistent problems of the traditional null hypothesis significance testing (NHST) that has been a mainstay of statistical inference for almost a century. 


Room
409