Bad Mood Rising? Assessing Scalar Invariance Violations with Comparative Democratic Support Data
2026-03-18
The advent of nearly global estimates of democratic mood has caused genuine optimism for comparative investigations into the linkages between public opinion and democracy. Scholarly enthusiasm in this field has particularly been boosted by recent claims that measuring latent democratic support with hierarchical IRT models overcomes differential item functioning (DIF)—a well-known challenge that typically foils the comparability of latent constructs across time and space. Focusing specifically on DIF-induced violations to scalar measurement invariance, we show mathematically and with statistical simulations that no commonly used latent variable modeling framework, including hierarchical IRT, is immune to bias stemming from systematic DIF. While some models can fully accommodate measurement invariance violations that are completely random between nations and across items, they begin to falter as soon as such violations exhibit a directional bias, that is, if respondents from different countries interpret or appraise survey items systematically differently. Equipped with democratic mood data from Latin America, we present suggestive evidence that systematic, directional bias in DIF is far more prevalent than random measurement noninvariance. We conclude with a number of practical recommendations for public opinion researchers to mitigate measurement invariance violations in their own work.