On the Limitations of the Neyman-Pearson, Likelihood Ratio, and Maximum Likelihood Criteria

Simple examples, both historical and recent, support the contention of Fisher and others that the NP criterion of a most powerful size alpha test is not, in general, relevant to the purposes of scientific inquiry. In these examples, the LR criterion provides appropriate inferences despite the existence of more powerful tests of the same size. In similar, equally simple examples, however, the LR and corresponding ML criteria provide inappropriate inferences. It is of interest to identify the distinguishing features of these two classes of examples, both of which involve only well-behaved statistical models, such as families of univariate normal distributions with known variances, and are not artifacts of irregularity, contamination, or unboundedness. The second class involves hypotheses consisting of the union of two or more subhypotheses of different linear dimensions, as may occur in model selection problems. This suggests the importance of determining the geometric nature of statistical models before routinely applying LRTs and MLEs. General LISREL models, including linear structural equations and latent variable models, may be problematic.