Background: Compared to children without special heath care needs, obtaining healthcare in adulthood uniquely challenges children with special health care needs (CSHCN). Because healthcare providers play an important role in helping mothers and CSHCN negotiate the adult system by introducing transition to adulthood concerns to them, the Maternal and Child Health Bureau (MCHB) monitors the extent to which CSHCN receive provider directed transition guidance. However, successful monitoring requires reliable measurement and no data describe the reliability of the question set used to estimate whether CSHCN receive guidance. If repeated under equivalent conditions, reliable measures provide similar results; they lead to reliable, accurate estimates. Unreliable questions do not. Without knowing the reliability of the question set, the extent to which CSHCN receive guidance about transitions remains obscure.
Study Question: To establish the reliability of MCHB’s transition to adulthood question set, we asked, “Can analysts use classical test theory (CTT) to describe the reliability of the transition to adulthood question set? And, if so, does the set provide reliable measurement?”
Methods: We used data from the 2005-2006 National Survey (NS) of CSHCN, a large (n = 40,840) nationally representative, randomly selected, telephone interviewed, sample of parents and guardians of CSHCN to explore the reliability of the transition to adulthood question set.
Results: CTT can psychometrically describe the items and it indicates that the “transition” question set provides marginal reliability (0.50). The Spearman-Brown Prophecy formula showed that adding three additional transition items would increase the questions set’s reliability to approximately 0.75. We did not explore whether the items provide better reliability between groups of children with more or less severe needs, which limits our findings, given that our results might not hold across the range of need severity.
Conclusions: The transition to adulthood question set does not provide sufficient reliability. NS-CSHCN estimates likely do not precisely monitor this outcome.
Public Health Implications: Policy makers and researchers monitoring CSHCN should express concern that estimates of CSHCN’s receipt of transition to adulthood guidance reflect large amounts of measurement error. The field should work to improve and increase the number of questions used to assess transition guidance before setting CSHCN policy using the current items.