A framework for assessing outcomes from newborn screening: on the road to measuring its promise.

UNLABELLED: Newborn screening (NBS) is intended to identify congenital conditions prior to the onset of symptoms in order to provide early intervention that leads to improved outcomes. NBS is a public health success, providing reduction in mortality and improved developmental outcomes for screened conditions. However, it is less clear to what extent newborn screening achieves the long-term goals relating to improved health, growth, development and function. We propose a framework for assessing outcomes for the health and well-being of children identified through NBS programs. The framework proposed here, and this manuscript, were approved for publication by the Secretary of Health and Human Services' Advisory Committee on Heritable Disorders in Newborns and Children (ACHDNC). This framework can be applied to each screened condition within the Recommended Uniform Screening Panel (RUSP), recognizing that the data elements and measures will vary by condition. As an example, we applied the framework to sickle cell disease and phenylketonuria (PKU), two diverse conditions with different outcome measures and potential sources of data. Widespread and consistent application of this framework across state NBS and child health systems is envisioned as useful to standardize approaches to assessment of outcomes and for continuous improvement of the NBS and child health systems. SIGNIFICANCE: Successful interventions for newborn screening conditions have been a driving force for public health newborn screening for over fifty years. Organizing interventions and outcome measures into a standard framework to systematically assess outcomes has not yet come into practice. This paper presents a customizable outcomes framework for organizing measures for newborn screening condition-specific health outcomes, and an approach to identifying sources and challenges to populating those measures.

A framework for key considerations regarding point-of-care screening of newborns.

Newborn screening is performed under public health authority, with analysis carried out primarily by public health laboratories or other centralized laboratories. Increasingly, opportunities to improve infant health will arise from including screening tests that are completed at the birth centers instead of in centralized laboratories, constituting a significant shift for newborn screening. This report summarizes a framework developed by the US Secretary of Health and Human Services Advisory Committee on Heritable Disorders in Newborns and Children based on a series of meetings held during 2011 and 2012. These meetings were for the purpose of evaluating whether conditions identifiable through point-of-care screening should be added to the recommended universal screening panel, and to identify key considerations for birth hospitals, public health agencies, and clinicians when point-of-care newborn screening is implemented.

Committee report: Method for evaluating conditions nominated for population-based screening of newborns and children.

The Secretary's Advisory Committee on Heritable Disorders in Newborns and Children is charged with evaluating conditions nominated for addition to the uniform screening panel and consequently making recommendations to the secretary of the US Department of Health and Human Services. This report describes the framework by which the committee approaches its task. Key decision nodes include initial review of every nomination to determine whether conditions are amenable for systematic evidence review, review of systematic evidence reviews conducted by the committee's external review group, and deliberation and formal recommendation for addition or exclusion to the uniform panel. Data analyzed include the accuracy and specificity of screening and diagnostic tests for nominated disorders, the extent of predicted health benefits, harms impact on disease course, and cost from early diagnosis and treatment. The committee process is guided by approaches used by similar entities, but more flexible criteria are sometimes needed to accommodate data limitations stemming from the rarity of many of these conditions. Possible outcomes of committee review range from recommendation to add a nominated condition to the uniform panel; provide feedback on specific gaps in evidence that must be addressed before making a decision; or rejection of a nomination (e.g., because of identified harms). The committee's structured evidence-based assessment of nominated conditions supports a consistently rigorous, iterative and transparent approach to its making recommendations regarding broad population-based screening programs for rare conditions in infants and children.

Long-term follow-up after diagnosis resulting from newborn screening: statement of the US Secretary of Health and Human Services' Advisory Committee on Heritable Disorders and Genetic Diseases in Newborns and Children.

The US Secretary of Health and Human Services' Advisory Committee on Heritable Disorders and Genetic Diseases in Newborns and Children provides guidance to reduce the morbidity and mortality associated with heritable disorders, with a special emphasis on those conditions detectable through newborn screening. Although long-term follow-up is necessary to maximize the benefit of diagnosis through newborn screening, such care is variable and inconsistent. To begin to improve long-term follow-up, the Advisory Committee has identified its key features, including the assurance and provision of quality chronic disease management, condition-specific treatment, and age-appropriate preventive care throughout the lifespan of affected individuals. There are four components central to achieving long-term follow-up: care coordination through a medical home, evidence-based treatment, continuous quality improvement, and new knowledge discovery.

Committee Report: advancing the current recommended panel of conditions for newborn screening.

The Advisory Committee on Heritable Disorders and Genetic Diseases in Newborns and Children is charged with advising the Secretary of the US Department of Health and Human Services in areas relevant to heritable conditions in children, especially newborn screening (NBS). This report describes the formulation by the Committee of a new process to nominate and review conditions to the recommended universal NBS panel. Nominations are currently being solicited. Committee review will adhere to the fundamental principles of being transparent, broadly accessible, evidence-based and consistent across the process for all of the proposed conditions across the process.

Study designs for effectiveness and translation research :identifying trade-offs.

BACKGROUND: Practitioners and policymakers need credible evidence of effectiveness to justify allocating resources to complex, expensive health programs. Investigators, however, face challenges in designing sound effectiveness and translation research with relevance for "real-world" settings. METHODS: Research experts and federal and foundation funders (n= approximately 120) prepared for and participated in a symposium, held May 4-5, 2004, to weigh the strengths, limitations, and trade-offs between alternate designs for studying the effectiveness and translation of complex, multilevel health interventions. RESULTS: Symposium attendees acknowledged that research phases (hypothesis generating, efficacy, effectiveness, translation) are iterative and cyclical, not linear, since research in advanced phases may reveal unanswered questions in earlier phases. Research questions thus always need to drive the choice of study design. When randomization and experimental control are feasible, participants noted that the randomized controlled trial with individual random assignment remains the gold standard for safeguarding internal validity. Attendees highlighted trade-offs of randomized controlled trial variants, quasi-experimental designs, and natural experiments for use when randomization or experimental control or both are impossible or inadequately address external validity. Participants discussed enhancements to all designs to increase confidence in causal inference while accommodating greater external validity. Since no single study can establish causality, participants encouraged replication of studies and triangulation using different study designs. Participants also recommended participatory research approaches for building population relevance, acceptability, and usefulness. CONCLUSIONS: Consideration of the study design choices, trade-offs, and enhancements discussed here can guide the design, funding, completion, and publication of appropriate policy- and practice-oriented effectiveness and translational research for complex, multilevel health interventions.

Road to the Future: Priorities for Child Health Services Research.

BACKGROUND: Prior health services research (HSR) agendas for children have been published, but major ones are now over 15 years old and do not reflect augmented understanding of the drivers and determinants of children's health; recent changes in the organization, financing, and delivery of health care; a growing emphasis on population health; and major demographic shifts in the population. A policy-relevant research agenda that integrates knowledge gained over the past 2 decades is essential to guide future child HSR (CHSR). We sought to develop and disseminate a robust, domestically focused, policy-oriented CHSR agenda. METHODS: The new CHSR agenda was developed through a series of consultations with leaders in CHSR and related fields. After each round of consultation, the authors synthesized the previous experts' guidance to help inform subsequent discussions. The multistep process in generation of the agenda included identification of major policy-relevant research domains and specification of high-value research questions for each domain. Stakeholders represented in the discussions included those with expertise in child and family advocacy, adult health, population health, community development, racial and ethnic disparities, women's health, health economics, and government research funders and programs. RESULTS: In total, 180 individuals were consulted in developing the research agenda. Six priority domains were identified for future research, including both enduring and emerging emphases: 1) framing children's health issues so that they are compelling to policy-makers; 2) addressing poverty and other social determinants of child health and wellbeing; 3) promoting equity in population health and health care; 4) preventing, diagnosing, and treating high priority health conditions in children; 5) strengthening performance of the health care system; and 6) enhancing the CHSR enterprise. Within these 6 domains, 40 specific topics were identified as the most pertinent for future research. Three overarching and crosscutting themes that affect research across the domains were also noted: the need for syntheses to build on the current, and sometimes extensive, evidence base to avoid duplication; the interrelated nature of the domains, which could lead to synergies in research; and the need for multidisciplinary collaborations in conducting research because research studies will look beyond the health sector. CONCLUSIONS: The priorities presented in the agenda are policy-oriented and include a greater emphasis on how findings are framed and communicated to support action. We expect that the agenda will be useful for immediate uptake by investigators and research funders.

Health care for children and youth in the United States: annual report on patterns of coverage, utilization, quality, and expenditures by a county level of urban influence.

OBJECTIVE: To examine child and hospital demographics and children's health care coverage, use, expenditures, and quality by a county-level measure of urban influence. METHODS: Two national health care databases serve as the sources of data for this report: the 2002 Medical Expenditure Panel Survey (MEPS) and the 2002 Nationwide Inpatient Sample (NIS) and State Inpatient Databases (SID) from the Healthcare Cost and Utilization Project (HCUP). In both data sets, county urbanicity is defined by use of a collapsed version of the 2003 Urban Influence Codes, to distinguish among children residing in and hospitals located in large metropolitan (metro) counties, small metro counties, micropolitan counties, and noncore counties. RESULTS: Demographics. In large metro counties, greater percentages of the child population are Hispanic or black non-Hispanic than in small metro, micropolitan, and noncore counties; in micropolitan and noncore counties, higher proportions of children are below 200% of the federal poverty level than in large metro and small metro counties. Noncore areas have a greater percentage of children in fair or poor health compared with those in small metro and micropolitan counties. Most hospitals are located in large and small metro areas, and large metro areas have a higher proportion of teaching hospitals compared with other areas. Health care. In general, there were no overall differences by place of residence in the proportion of children with and without insurance, although differences emerged in subpopulations within Urban Influence Code types. Hispanic children residing in large metro counties were more likely to be uninsured than those in small metro counties. Overall, the proportion of children with at least one dental visit was larger in small metro areas compared with both large metro and noncore areas. The proportion of children with medicines prescribed was generally lower in large metro areas compared with all other areas both overall and among subpopulations of children. Children in noncore areas were more likely to have a hospital inpatient stay and any emergency department use compared with children in large metro areas. Children in large metro counties had longer average inpatient stays and a higher hospital inpatient charge per day compared with children in all other counties. Although most hospitalizations for children from large metro areas occurred in large metro areas, over half of hospitalizations for noncore children occurred outside of noncore counties. Further, children from noncore counties appear to be hospitalized for ambulatory sensitive conditions more than children from all other areas. CONCLUSIONS: County-level data analyses performed using a collapsed version of the Urban Influence Codes with MEPS and HCUP data shed additional light on the health care patterns for children that were not previously evident when only the dichotomous metropolitan/nonmetropolitan geographic schema was used.

State challenges to child health quality measure reporting and recommendations for improvement.

OBJECTIVE: The Children's Health Insurance Program (CHIP) was re-authorized in 2009, ushering in an unprecedented focus on children's health care quality one of which includes identifying a core set of performance measures for voluntary reporting by states' Medicaid/CHIP programs. However, there is a wide variation in the quantity and quality of measures states chose to report to the Center's for Medicare & Medicaid Services (CMS). The objective of this study is to assess reporting barriers and to identify potential opportunities for improvement. METHODS: From 2013 to 2014 a questionnaire developed in coordination with CMS and the Agency for Healthcare Research and Quality (AHRQ) was sent to state Medicaid and CHIP officials to assess barriers to child health quality reporting for Federal Fiscal Year 2012. States were categorized as high, medium, or low reporting for comparative analysis. RESULTS: Twenty-five of the 50 states and the District of Columbia agreed to participate in the study and completed the questionnaire. States placed a high priority on children's health care quality reporting (4.2 of 5 point Likert Scale, SD 0.99) and 96% plan to use measurement results to further improve their quality initiatives. However, low reporting states believed they had inadequate staffing and that data collection and extraction was too time-consuming than high reporting states. CONCLUSION: Based on state responses, possible solutions to improve reporting includes funding and staff support, refining the technical assistance provided, and creating venues for state-to-state interaction. Realistic and tangible improvements are within reach and opportunities for CMS and states to collaborate to improve child health care quality.

Health care for children and youth in the United States: annual report on patterns of coverage, utilization, quality, and expenditures by income.

OBJECTIVES: To examine differences by income in insurance coverage, health care utilization, expenditures, and quality of care for children in the United States. METHODS: Two national health care databases serve as the sources of data for this report: the 2000-2002 Medical Expenditure Panel Survey (MEPS) and the 2001 Nationwide Inpatient Sample (NIS) from the Healthcare Cost and Utilization Project (HCUP). In the MEPS analyses, low income is defined as less than 200% of the federal poverty level and higher income is defined as 200% of the federal poverty level or more. For the HCUP analyses, median household income for the patient's zip code of residence is used to assign community-level income to individual hospitalizations. RESULTS: Coverage. Children from low-income families were more likely than children from middle-high-income families to be uninsured (13.0% vs 5.8%) or covered by public insurance (50.8% vs 7.3%), and less likely to be privately insured (36.2% vs 87.0%). Utilization. Children from low-income families were less likely to have had a medical office visit or a dental visit than children from middle-high-income families (63.7% vs 76.5% for office-based visits and 28.8% vs 51.4% for dental visits) and less likely to have medicines prescribed (45.1% vs 56.4%) or have utilized hospital outpatient services (5.2% vs 7.0%), but more likely to have made trips to the emergency department (14.6% vs 11.4%). Although low-income children comprise almost 40% of the child population, one quarter of total medical expenditures were for these children. Hospital Discharges. Significant differences by community-level income occurred in specific characteristics of hospitalizations, including admissions through the emergency department, expected payer, mean total charges per day, and reasons for hospital admission. Leading reasons for admission varied by income within and across age groups. Quality. Low-income children were more likely than middle-high-income children to have their parents report a big problem getting necessary care (2.4% vs 1.0%) and getting a referral to a specialist (11.5% vs 5.3%). Low-income children were at least twice as likely as middle-high-income children to have their parents report that health providers never/sometimes listened carefully to them (10.0% vs 5.1%), explained things clearly to the parents (9.6% vs 3.4%), and showed respect for what the parents had to say (9.2% vs 4.2%). Children from families with lower community-level incomes were more likely to experience ambulatory-sensitive hospitalizations. Racial/Ethnic Differences Between Income Groups. Use and expenditure patterns for most services were not significantly different between low- and middle-high-income black children and were lower than those for white children. CONCLUSIONS: While health insurance coverage is still an important factor in obtaining health care, the data suggest that efforts beyond coverage may be needed to improve access and quality for low-income children overall and for children who are racial and ethnic minorities, regardless of income.

The importance of outcomes research in pediatric emergency medicine.

Applying the methods and tools of outcomes research, "evaluation of the impact of health care on the health outcomes or 'end results' of patients and populations," to the clinical domain of emergency services for children offers an important strategic opportunity for addressing the questions that confront all health care services: What works? For which patients? At what cost? From whose perspective? Although the important questions to address are extensive, much of the intersection between emergency services and outcomes research remains unexplored. Important challenges for researchers intrigued by the opportunities at this intersection of fields include: 1) clear definition of the scope of emergency services; 2) consideration of the appropriate end-point of emergency services-the entire episode of illness and/or services provided within the emergency setting; 3) selection and development of measures that incorporate children's and families' perspectives; and 4) a clear focus on linking research findings with strategies for improving outcomes and informed decision making. This essay will provide an overview of accomplishments and challenges from the field of outcomes research, suggest important opportunities for applying existing methods to emergency medical services for children, and identify potential career paths for current and future investigators.

Health care for children and youth in the United States: 2002 report on trends in access, utilization, quality, and expenditures.

OBJECTIVE: To examine changes in insurance coverage, health care utilization, perceived quality of care, and expenditures for children and youth in the United States using data from 1987-2001. METHODS: Three national health care databases serve as the sources of data for this report. The Medical Expenditure Panel Survey (1996-2001) provides data on insurance coverage, utilization, expenditures, and perceived quality of care. The National Medical Expenditure Survey (1987) provides additional data on utilization and expenditures. The Nationwide Inpatient Sample (1995-2000) from the Healthcare Cost and Utilization Project provides information on hospitalizations. RESULTS: The percent of children uninsured for an entire year declined from 10.4% in 1996 to 7.7% in 1999. Most changes in children's health care occurred between 1987 and the late 1990s. Overall utilization of hospital-based services has declined significantly since 1987, especially for inpatient hospitalization. Several of the observed changes from 1987 varied significantly by type of health insurance coverage, poverty status, and geographic region. Quality of care data indicate some improvement between 2000 and 2001, which varies by insurance coverage. Overall, mean length of stay of hospitalizations did not change significantly from 1995 to 2000, but changes in the prevalence of hospitalizations and the length of stay associated with age-specific diagnoses were evident during this time period. CONCLUSIONS: Health care for children and youth has changed significantly since 1987, with most of the changes occurring between 1987 and 1996. Insurance coverage has improved, the site of care has shifted toward ambulatory sites, hospital utilization has declined, and expenditures on children as a proportion of total expenditures have decreased. Variation in these changes is evident by insurance status, poverty, and region.

Child and adolescent health care quality and disparities: are we making progress?

OBJECTIVE: Children and adolescents are known to experience poor health care quality; some groups of children have poorer health care than others. We sought to examine trends over time in health care quality and disparities by race, Hispanic ethnicity, income, insurance, gender, rurality, and special health care needs. METHODS: Source data were extracted from the 2011 National Healthcare Quality Report (NHQR) and National Healthcare Disparities Report (NHDR) database, which contains aggregated data from many government and private sources for the years 2000 through 2009. The NHQR and NHDR approaches to calculating disparities and trends in quality and disparities were used. Within each quality measure with available data, results for demographic subgroups of children characterized by race/ethnicity, income, insurance, residence, special health care need, and gender were compared to those of a reference group to determine whether disparities existed and whether disparities had changed over time. RESULTS: Of 68 measures with data for calculating potential disparities, 50 showed disparities in quality for at least 1 comparison subgroup in the most recent year of data available, while 18 measures showed no such disparities. Of the 50 measures with current disparities, 39 measures had sufficient data to calculate trends. Among the 137 comparisons made within these 39 measures, there was no change in disparities over time for 126 comparisons, 3 comparisons worsened, and 8 comparisons improved. CONCLUSIONS: There was some progress in health care quality and reducing disparities in children's health care quality from 2000 to 2009; opportunities for targeting improvement strategies remain.

Advancing children's health care and outcomes through the pediatric quality measures program.

In 2009 Congress passed the Children's Health Insurance Program Reauthorization Act (CHIPRA), which presented an unprecedented opportunity to measure and improve health care quality and outcomes for children. The Agency for Healthcare Research and Quality, in partnership with the Centers for Medicare & Medicaid Services, has worked to fulfill a number of quality measurement provisions under CHIPRA, including establishing the Pediatric Quality Measures Program (PQMP). The PQMP was charged with establishing a publicly available portfolio of new and enhanced evidence-based pediatric quality measures for use by Medicaid/Children's Health Insurance Program and other public and private programs and to also provide opportunities to improve and strengthen the Child Core Set of quality measures. This article focuses on the PQMP and provides an overview of the program's goals and related activities, lessons learned, and future opportunities.

Systematic evidence-based quality measurement life-cycle approach to measure retirement in CHIPRA.

OBJECTIVE: In 2009, Centers for Medicare and Medicaid Services (CMS) publicly released an initial child core set (CCS) of health care quality measures for voluntary reporting by state Medicaid and Children's Health Insurance Program (CHIP) programs. CMS is responsible for implementing the reporting program and for updating the CCS annually. We assessed selected CCS measures for potential retirement. METHODS: We identified a 23-member external advisory group to provide relevant expertise. We worked with the group to identify 4 major criteria with multiple subcomponents for assessing the measures. We provided information corresponding to each criterion and subcriterion, using a variety of sources such as the 2009 Medicaid Analytic eXtract (MAX), state-level Medicaid and CHIP data submitted to the CMS, and summaries of published literature on clinical and quality improvement effectiveness related to the CCS topics. Using this information, the group: 1) used a modified Delphi process to score the measures in 2 anonymous scoring rounds (on a scale of 1 to 9 in each round); 2) voted on whether each measure should be retired; and 3) provided narrative explanations of their choices (which formed the basis of our qualitative findings). Recommendations were reviewed by CMS before promulgation to state programs. RESULTS: The Subcommittee of the National Advisory Council on Healthcare Research and Quality (SNAC) recommended that the 4 major criteria be importance, scientific acceptability, feasibility, and usability. The SNAC recommended 3 measures for retirement: access to primary care; testing for strep before recommending antibiotics for pharyngitis; and annual HbA1c testing of children with diabetes. Explanations for suggesting retirement of the measures included: views that the well-visit measures were a better measure of access than the primary care measure; a likely ceiling effect (pharyngitis); and the paucity of clinical evidence and low prevalence (both for HbA1c). CMS recommended that state Medicaid and CHIP programs retire 2 of the recommended measures from the CCS, but retained the access to primary care measure. CONCLUSIONS: Periodic reassessment of the value of health care quality measures can reduce reporting burden and allow measure users to focus on measures with higher likelihood of leading to improvements in quality of care and child health outcomes.

Assessing quality improvement in health care: theory for practice.

OBJECTIVES: To review the role of theory as a means to enhance the practice of quality improvement (QI) research and to propose a novel conceptual model focused on the operations of health care. METHODS: Conceptual model, informed by literature review. RESULTS: To optimize learning across QI studies requires the integration of small-scale theories (middle-range theories, theories of change) within the context of larger unifying theories. We propose that health care QI research would benefit from a theory that describes the operations of health care delivery, including the multiplicity of roles that interpersonal interactions play. The broadest constructs of the model are entry into the system, and assessment and management of the patient, with the subordinate operations of access; recognition, assessment, and diagnosis; and medical decision-making (developing a plan), coordination of care, execution of care, referral and reassessment, respectively. Interpersonal aspects of care recognize the patient/caregiver as a source of information, an individual in a cultural context, a complex human being, and a partner in their care. Impacts to any and all of these roles may impact the quality of care. CONCLUSIONS: Such a theory can promote opportunities for moving the field forward and organizing the planning and interpretation of comparable studies. The articulation of such a theory may simultaneously provide guidance for the QI researcher and an opportunity for refinement and improvement.

Pediatric health care quality measures: considerations for pharmacotherapy.

Measuring the quality use of medicines can be conceptualized as a mechanism for understanding appropriate use, underuse, overuse, or misuse. For pediatric pharmacotherapy, measuring the quality use of medicines requires awareness of the differences in health care between children and adults and the differences in the quality and quantity of science that supports evidence-based practice in pediatric health care compared with adult health care. Here we use the Pediatric Quality Measures Program that arose from the Children's Health Insurance Program Reauthorization Act in the United States to illustrate the challenges in developing quality measures of pediatric pharmacotherapy. The challenges are primarily twofold: (i) weak evidence base for the specific pharmacotherapy in children and (ii) limited data to calculate the measure. A weak evidence base must often be weighed against the importance of the topic if the quality measure is intended to address a known quality of care or public health problem. Limited data because of insufficient amount or inappropriate type will affect implementation of the measure and its eventual usefulness. Methods to meet these challenges often depend on the priorities of and the tools available to end users. Health information technology is emerging as a tool to improve quality measurement but presents additional challenges.