Beyond Right and Wrong: A Spatial Approach to Categorically Scored Objective Items

Problem: In traditional educational and intelligence tests examinees' answers are either right or wrong. The same is true in many other rating systems. For example, various political activist groups will rate legislators as voting for the "hawk" vs. "dove" position, or pro-environmentalist vs. pro-business. We are investigating an alternative technique, in which multiple responses to the same item can be rated as being associated with various positions, or underlying ideas called facets, without a necessary implication that ideas are right or wrong. Although this scoring system has several educational advantages, it does not lend itself to conventional psychometric applications. We will describe a newly developed psychometric approach. The application will be illustrated with examples in which high school and middle school students respond to material taught in the physical science curriculum. We discuss the relation of our approach to the conventional and to another alternative.