In a predictive model, what is the expected change in the outcome associated with a unit change in one of the inputs? In a linear regression model without interactions, this "average predictive effect" is simply a regression coefficient (with associated uncertainty). In a model with nonlinearity or interactions, however, the average predictive effect in general depends on the values of the predictors. We consider various definitions based on averages over a population distribution of the predictors, and we compute standard errors based on uncertainty in model parameters. We illustrate with a study of criminal justice data for urban counties in the United States. The outcome of interest measures whether a convicted felon received a prison sentence rather than a jail or non-custodial sentence, with predictors available at both individual and county levels. We fit three models: a hierarchical logistic regression with varying coefficients for the within-county intercepts as well as for each individual predictor; a hierarchical model with varying intercepts only; and a non-hierarchical model that ignores the multilevel nature of the data. The regression coefficients have different interpretations for the different models; in contrast, the models can be compared directly using predictive effects. Furthermore, predictive effects clarify the interplay between the individual and county predictors for the hierarchical models, as well as illustrating the relative size of varying county effects.
This work is joint with Andrew Gelman, Department of Statistics, Columbia University
