Skip to main content

Robustly Estimating Heterogeneity in Factorial Data Using Rashomon Partitions

Tyler McCormick Headshot

Tyler McCormick, Professor, Statistics & Sociology, University of Washington

Abstract:

Many statistical analyses, in both observational data and randomized control trials, ask: how does the outcome of interest vary with combinations of observable covariates? How do various drug combinations affect health outcomes, or how does technology adoption depend on incentives and demographics? Our goal is to partition this factorial space into "pools" of covariate combinations where the outcome differs across the pools (but not within a pool). Existing approaches (i) search for a single "optimal" partition under assumptions about the association between covariates or (ii) sample from the entire set of possible partitions. Both these approaches ignore the reality that, especially with correlation structure in covariates, many ways to partition the covariate space may be statistically indistinguishable, despite very different implications for policy or science. We develop an alternative perspective, called Rashomon Partition Sets (RPSs). Each item in the RPS partitions the space of covariates using a tree-like geometry. RPSs incorporate all partitions that have posterior values near the maximum a posteriori partition, even if they offer substantively different explanations, and do so using a prior that makes no assumptions about associations between covariates. This prior is the ℓ0 prior, which we show is minimax optimal. Given the RPS we calculate the posterior of any measurable function of the feature effects vector on outcomes, conditional on being in the RPS. We also characterize approximation error relative to the entire posterior and provide bounds on the size of the RPS. Simulations demonstrate this framework allows for robust conclusions relative to conventional regularization techniques. We apply our method to three empirical settings: price effects on charitable giving, chromosomal structure (telomere length), and the introduction of microfinance.

 

Tyler's work develops statistical models for inference and prediction in scientific settings where data are sparsely observed or measured with error. His recent projects include estimating features of social networks (e.g. the degree of clustering or how central an individual is) using data from standard surveys, inferring a likely cause of death (when deaths happen outside of hospitals) using reports from surviving caretakers, and quantifying & communicating uncertainty in predictive models for global health policymakers. He holds a Ph.D. in Statistics (with distinction) from Columbia University and is the recipient of the NIH Director's New Innovator Award, NIH Career Development (K01) Award, Army Research Office Young Investigator Program Award, and a Google Faculty Research Award. Currently, he is a Professor of Statistics and Sociology at the University of Washington, where he is also a core faculty member in the Center for Statistics and the Social Sciences and a Senior Data Science Fellow at the eScience Institute. During the 2019-2020 academic year Tyler was a Visiting Faculty Researcher at Google People+AI Research (PAIR).  Tyler is the former Editor of the Journal of Computational and Graphical Statistics (JCGS) and a Fellow of the American Statistical Association.  

 


Room
409