Estimation of stochastic scale with best-worst data

Abstract

Recently there has been a steady stream of literature advocating the best-worst response mechanism, where respondents are asked to sequentially choose the best and worst alternatives in a choice set, resulting in a partial or complete ranking of the alternatives. In this paper, we present an empirical study in which respondents were encouraged to respond using repeated best-worst, but were nonetheless free to respond in any order they preferred. Models are estimated that account for three alternative response processes: conventional ranking of the alternatives from best to worst, sequential best-worst choice, and two best choices followed by two worst choices. While the sequential best-worst models perform best, the sensitivities retrieved are consistent across all three models. We find strong evidence of stochastic scale heterogeneity across respondents, where the extent of this heterogeneity is also consistent across all three model forms. However, deterministic scale heterogeneity, that accounts for differences in scale across each of the pseudo-observations, is not consistent across the model forms, with respect to the implied rank of the observation. Rather, the consistency is with the number of alternatives associated with the pseudo-observation, with scale decreasing as the number of alternatives decreases. A test of alternative specifications of the panel in the mixture model used to identify stochastic scale identifies that scale should be invariant across the full set of responses by an individual, rather than just the responses from each rank from that individual. Despite an overall finding that the sensitivities retrieved are robust to the assumption of the completion order of the ranking within the model, differences in sensitivities retrieved from each best-worst choice raise concerns with pooling the data across best-worst choices, in line with concerns raised previously with rankings data.