Specification tests are designed to assess the validity of the assumptions underlying the model being estimated before the model can be considered as adequate. Common examples of specification tests include White's IM test, Ramsey's RESET test, and the Hausman-McFadden test of IIA for multinomial logit. In practice, it is tempting for a data analyst to use such tests as part of a specification search. If the hypothesis being tested is not rejected, estimates of the current specification are used; if the hypothesis is rejected, the model is re-specified to address the failing suggested by the test. In the paper, I present a series of Monte Carlo simulations to illustrate the broad limitations of specification tests. These include: 1) results that are highly sensitive to what are essentially arbitrary decisions in constructing the test; 2) sensitivity of tests to violations of assumptions other than those explicitly being tested; 3) poor size properties even in large samples; 4) size properties that are highly dependent upon characteristics of the data rather than violations of the assumptions being tested; and 5) poor properties when tests are used for screening, with the final model modified based on the prior screening. Overall, the results suggest that using specification tests to select a final model often lead to more problems than solutions.
© 2024 Center for Statistics and the Social Sciences