Importance There is increased interest in public reporting of, and linking financial incentives to, the performance of organizations on health equity metrics, but variation across organizations could reflect differences in performance or selection bias.
Objective To assess whether differences across health plans in sex- and age-adjusted racial disparities are associated with performance or selection bias.
Design, Setting, and Participants This cross-sectional study leveraged a natural experiment, wherein a southern US state randomly assigned much of its Medicaid population to 1 of 5 plans after shifting to managed care in 2012. Enrollee-level administrative claims and enrollment data from 2011 to 2015 were obtained for self-identified Black and White enrollees. The analyses were limited to Black and White Medicaid enrollees because they accounted for the largest percentages of the population and could be compared with greater statistical power than other groups. Data were analyzed from June 2021 to September 2024.
Exposures Plan enrollment via self-selection (observational population) vs random assignment (randomized population).
Main Outcomes and Measures Annual counts of primary care visits, low-acuity emergency department visits, prescription drug fills, and total spending. For observational and randomized populations, models of each outcome were fit as a function of plan indicators, indicators for race, interactions between plan indicators and race, and age and sex. Models estimated the magnitude of racial differences within each plan and tested whether this magnitude varied across plans.
Results Of 118 101 enrollees (mean [SD] age, 9.3 [7.5] years; 53.0% female; 61.4% non-Hispanic Black; and 38.6% non-Hispanic White), 70.2% were included in the randomized population, and 29.8% were included in the observational population. Within-plan differences in primary care visits, low-acuity emergency department visits, prescription drug use, and total spending between Black and White enrollees were large but did not vary substantially and were not statistically significantly different across plans in the randomized population, suggesting minimal effects of plans on racial differences in these measures. In contrast, in the observational population, racial differences varied substantially across plans (standard deviations 2-3 times greater than in the randomized population); this variation was statistically significant after adjustment for multiple testing, except for emergency department visits. Greater between-plan variation in racial differences in the observational population was only partially explained by sampling error. Stratifying by race did not bring observational estimates of plan effects meaningfully closer to randomized estimates.
Conclusions and Relevance This cross-sectional study showed that selection bias may mischaracterize plans’ relative performance on measures of health care disparities. It is critical to address disparities in Medicaid, but adjusting plan payments based on disparity measures may have unintended consequences.