Exploring the Effects of Item-Specific Factors in Tree-Based Item Response Models

Abstract:

Item response theory (IRT) is currently the dominant methodological paradigm in educational and psychological measurement. IRT models assume that each respondent has their own latent trait, conditioning on which their observed responses are independent discrete random variables. Recently, much attention has been given to tree-based methods for their ability to model test items whose scores reflect the outcomes of underlying multi-step psychological processes. Despite the presence of multiple stages within the same item, conditional independence is still assumed for estimation purposes. However, we argue that in practice there is often good reasons to suspect the existence of shared item-specific factors across stages within each item. Although not statistically detectable, omission of such factors leads to the missing not at random condition and creates ambiguity in the interpretations of IRT model parameters. In this talk, I will provide a brief overview of traditional and tree-based IRT models, show the consequences of omitting item-specific factors through simulation, and discuss implications in relation to some applications in previous literature. I will conclude the talk by pointing out some possible future directions, including one of my ongoing projects which extends the item-specific factor framework to continuous outcomes such as response times.


Weicong Lyu is a postdoctoral scholar in the Measurement and Statistics program within the College of Education at the University of Washington. He received his Ph.D. in Educational Psychology, M.S. in Computer Science, and M.A. in Mathematics from the University of Wisconsin-Madison. His research interests include item response theory, Bayesian methods and causal inference.