Predictors of Subspecialty Appointment Scheduling and Completion for Patients Referred From a Pediatric Primary Care Clinic.

Failure to complete subspecialty referrals decreases access to subspecialty care and may endanger patient safety. We conducted a retrospective analysis of new patient referrals made to the 14 most common referral departments at Boston Children's Hospital from January 1 to December 31, 2017. The sample included 2031 patient referrals. The mean wait time between referral and appointment date was 39.6 days. In all, 87% of referrals were scheduled and 84% of scheduled appointments attended, thus 73% of the original referrals were completed. In multivariate analysis, younger age, medical complexity, being a non-English speaker, and referral to a surgical subspecialty were associated with a higher likelihood of referral completion. Black and Hispanic/Latino race/ethnicity, living in a Census tract with Social Vulnerability Index (SVI) ≥ 90th percentile, and longer wait times were associated with a lower likelihood of appointment attendance. Future interventions should consider both health care system factors such as appointment wait times and community-level barriers to referral completion.

How much did it cost to develop and implement an eHealth intervention for a minority children population that overlapped with the COVID-19 pandemic?

BACKGROUND: eHealth interventions using active video games (AVGs) offer an alternative method to help children exercise, especially during a pandemic where options are limited. There is limited data on costs associated with developing and implementing such interventions. OBJECTIVES: We quantified the costs of delivering an eHealth RCT intervention among minority children during COVID-19. METHODS: We categorized the total trial cost into five subcategories: intervention material development, advertising and recruitment, intervention delivery, personnel salaries, and COVID-19-related equipment costs. RESULTS: The total RCT cost was $1,927,807 (Direct: $1,227,903; Indirect: $699,904) with three visits required for each participant. The average cost per participant completing the RCT (79 participants/237 visits) was $24,403 (Direct: $15,543; Indirect: $8860). Due to no-shows and cancellations (198 visits) and dropouts before study completion (61 visits; 56 participants), 496 visits had to be scheduled to ensure complete data collection on 79 participants. If all 496 visits were from participants completing the three-visit protocol, that would correspond to 165 participants, bringing the average cost per participant down to $11,684 (Direct: $7442; Indirect: $4242). Of the subcategories, intervention material development accounted for the largest portion, followed by personnel salaries. While the direct COVID-19-specific cost constituted <1% of the entire budget, the indirect effects were much larger and significantly impacted the trial. CONCLUSION: RCTs typically involve significant resources, even more so during a pandemic. Future eHealth intervention investigators should budget and plan accordingly to prepare for unexpected costs such as recruitment challenges to increase flexibility while maximizing the intervention efficacy.

Improving HPV Vaccination Rates in a Racially and Ethnically Diverse Pediatric Population.

BACKGROUND AND OBJECTIVES: Nationally, 54.2% of youth are fully vaccinated for human papilloma virus (HPV) with persistent gender and racial/ethnic disparities. We used a quality improvement approach to improve completion of the HPV vaccine series by age 13 years. As a secondary aim, we examined racial/ethnic and gender differences in vaccine uptake. METHODS: The study setting included 2 pediatric, academic, primary care practices in Massachusetts. We designed a multilevel patient-, provider-, and systems-level intervention addressing parental hesitancy, provider communication, and clinical operations. Rates of HPV series completion by age 13 were monitored using a control p chart. Bivariate and multivariate analyses evaluated vaccine completion differences on the basis of clinic size, gender, and race/ethnicity. RESULTS: Between July 1, 2014, and September 30, 2021, control p charts showed special cause variation with HPV vaccine initiation by age 9 years, increasing from 1% to 52%, and vaccine completion by 13 years, increasing from 37% to 77%. Compared with White and Black children, Hispanic children were more likely to initiate the HPV vaccine at age 9 (adjusted odds ratio [95% confidence interval] = (1.4-2.6)] and complete the series by age 13 (adjusted odds ratio [95% confidence interval] = 2.3 (1.7-3.0). CONCLUSIONS: A multilevel intervention was associated with sustained HPV vaccine series completion by age 13 years. Hispanic children were more likely to be vaccinated. Qualitative family input was critical to intervention design. Provider communication training addressed vaccine hesitancy. Initiation of the vaccine at age 9 and clinicwide vaccine protocols were key to sustaining improvements.

Effect of Computer-Based Substance Use Screening and Brief Behavioral Counseling vs Usual Care for Youths in Pediatric Primary Care: A Pilot Randomized Clinical Trial.

Importance: Annual preventive health visits provide an opportunity to screen youths for unhealthy substance use and intervene before serious harm results. Objectives: To assess the feasibility and acceptability and estimate the efficacy of a primary care computer-facilitated screening and practitioner-delivered brief intervention (CSBI) system compared with usual care (UC) for youth substance use and associated risk of riding with an impaired driver. Design, Setting, and Participants: An intent-to-treat pilot randomized clinical trial compared CSBI with UC among 965 youths aged 12 to 18 years at 5 pediatric primary care offices and 54 practitioners. Patients were randomized to CSBI (n = 628) or usual care (n = 243) groups within practitioner with 12 months of follow-up. Data were collected from February 1, 2015, to December 31, 2017. Data analysis was performed January 1, 2018, to March 30, 2019. Interventions: Patients self-administered a computer-facilitated substance use screening questionnaire before their annual preventive health visits. Immediately after completing the screening, they received their score and level of risk and viewed 10 pages of scientific information and true-life vignettes illustrating health risks associated with substance use. Trained practitioners received the screening results, patients' risk levels, talking points designed to prompt brief counseling, and recommended follow-up plans. Main Outcomes and Measures: Feasibility and acceptability were assessed using adolescents' postvisit ratings. Days of alcohol use, cannabis use, and heavy episodic drinking were assessed at baseline and 3-, 6-, 9-, and 12-month follow-ups using Timeline Followback, and riding in the past 3 months with a driver who was impaired by use of alcohol or other drugs was assessed using 2 self-report items. The primary outcome was the intervention effect among at-risk youths who reported using alcohol or other drugs in the past 12 months or riding with an impaired driver in the past 3 months at baseline. The secondary outcome was the prevention effect among those with no prior use or risk. Results: Among 871 youths screened, 869 completed the baseline assessment; 211 of the 869 reported alcohol or cannabis use in the past 12 months at baseline (mean [SD] age, 16.4 [1.3] years; 114 [54.1%] female; 105 [49.8%] non-Hispanic white). Of the 211 youths, 148 (70.1%) were assigned to the CSBI group and 63 (29.9%) were assigned to the UC group. Among youths in the CSBI group, 105 (70.9%) reported receiving counseling about alcohol, 122 (82.4%) reported receiving counseling about cannabis, and 129 (87.2%) reported receiving counseling about not riding with an impaired driver. Adjusted hazard ratios for time to first postvisit use of alcohol or other drugs for CSBI vs UC were as follows: alcohol use, 0.69 (95% CI, 0.47-1.02); heavy episodic drinking, 0.66 (95% CI, 0.40-1.10); and cannabis use, 0.62 (95% CI, 0.41-0.94). At 12-month follow-ups among 99 youths who reported having ridden in the past 3 months at baseline with an impaired driver (64 in the CSBI group; 35 in the UC group), adjusted relative risk ratio of riding in the past 3 months with an impaired driver for CSBI vs UC groups was 0.58 (95% CI, 0.37-0.91). No intervention effect was observed among youths who reported no prior use of alcohol or other drugs (n = 658) or not having ridden with an impaired driver (n = 769) at baseline. Conclusions and Relevance: The CSBI system is a feasible and acceptable option for screening youths in primary care practice for use of alcohol and other drugs and for risk of riding with an impaired driver, and the estimated efficacy in this sample warrants further testing in larger samples. Trial Registration: ClinicalTrials.gov identifier: NCT00227877.

Resident Experience and Education in Academic and Community Pediatric Primary Care Sites: Lessons for a Changing Healthcare Landscape.

INTRODUCTION: Efforts to improve pediatric primary care training in residency are important both for the residents and for the patients cared for in residency clinics. Pediatric residents typically get their primary care training at primary care centers affiliated with an academic center or at community-based locations. We aimed to compare residents' experience of continuity clinic in academic centers and community settings, and to identify relative strengths and weaknesses of each. METHODS: Survey data was evaluated for residents at one large pediatric residency program. RESULTS: Community sites had relative strengths in patient flow, population management, and perception of overall quality of care. Academic sites had relative strengths in continuity of care and ease of follow-up of results. CONCLUSIONS: Community and academic pediatric primary care training sites have varied strengths that could inform efforts to improve residency training to better meet the needs of residents and patients.

Shared Care: Using an Electronic Consult Form to Facilitate Primary Care Provider-Specialty Care Coordination.

OBJECTIVE: The quality of children's health is compromised by poor care coordination between primary care providers (PCPs) and specialists. Our objective was to determine how an electronic consultation and referral system impacts referral patterns and PCP-specialist communication. METHODS: The primary care clinic at Boston Children's Hospital piloted an electronic referral and consultation system with the neurology and gastroenterology departments from April 1, 2014, to October 31, 2016. PCPs completed an electronic consult form, and if needed, specialists replied with advice or facilitated expedited appointments. Specialist response times, referral rates, wait times, and completion rates for specialty visits were tracked. PCPs and specialists also completed a survey to evaluate feasibility and satisfaction. RESULTS: A total of 82 PCPs placed 510 consults during the pilot period. Specialists responded to 88% of requests within 3 business days. Eighteen percent of specialty visits were deferred and 21% were expedited. Wait times for specialty appointments to both departments significantly decreased, from 48 to 34 days (P < .001), and completion rates improved from 58% to 70% (P < .01), but referral volumes remained stable (25 per month to 23 per month; P = .29). Most PCPs said the Shared Care system facilitated better communication with specialists (89%) and enabled them to provide superior patient care (92%). Specialists reported that the system required a minimal amount of time and enabled them to educate PCPs and triage referrals. CONCLUSIONS: Implementation of an electronic referral and consultation system was feasible and provided timely access to specialty care, but did not affect referral volume. This system could serve as a model for other health care organizations and specialties.

Barriers to HPV immunization among blacks and latinos: a qualitative analysis of caregivers, adolescents, and providers.

BACKGROUND: Despite recommendations that 11-12-year-olds receive the full three-shot Human papillomavirus (HPV) vaccine series, national HPV immunization coverage rates remain low. Disparities exist, with Blacks and Latinos being less likely than Whites to complete the series. We aimed to identify and compare barriers to HPV immunization perceived by healthcare providers, Black and Latino adolescents, and their caregivers to inform a clinic-based intervention to improve immunization rates. METHODS: We conducted semi-structured interviews between March and July 2014 with Black and Latino adolescents (n = 24), their caregivers (n = 24), and nurses (n = 18), and 2 focus groups with 18 physicians recruited from two pediatric primary care clinics. Qualitative protocol topics included: general perceptions and attitudes towards vaccines; HPV knowledge; and perceived individual and systems-level barriers affecting vaccine initiation and completion. RESULTS: Themes were identified and organized by individual and systems-level barriers to HPV immunization. Adolescents and their caregivers, particularly Blacks, expressed concerns about HPV being an untested, "newer" vaccine. All families felt they needed more information on HPV and found it difficult to return for multiple visits to complete the vaccine series. Providers focused on challenges related to administering multiple vaccines simultaneously, and perceptions of parental reluctance to discuss sexually transmitted infections. CONCLUSIONS: Optimizing HPV immunization rates may benefit from a multi-pronged approach to holistically address provider, structural, and individual barriers to care. Further research should examine strategies for providing multiple modalities of support for providers, including a routinized system of vaccine promotion and delivery, and for addressing families' concerns about vaccine safety and efficacy.

In vitro cytokine induction by TLR-activating vaccine adjuvants in human blood varies by age and adjuvant.

Most infections occur in early life, prompting development of novel adjuvanted vaccines to protect newborns and infants. Several Toll-like receptor (TLR) agonists (TLRAs) are components of licensed vaccine formulations or are in development as candidate adjuvants. However, the type and magnitude of immune responses to TLRAs may vary with the TLR activated as well as age and geographic location. Most notably, in newborns, as compared to adults, the immune response to TLRAs is polarized with lower Th1 cytokine production and robust Th2 and anti-inflammatory cytokine production. The ontogeny of TLR-mediated cytokine responses in international cohorts has been reported, but no study has compared cytokine responses to TLRAs between U.S. neonates and infants at the age of 6months. Both are critical age groups for the currently pediatric vaccine schedule. In this study, we report quantitative differences in the production of a panel of 14 cytokines and chemokines after in vitro stimulation of newborn cord blood and infant and adult peripheral blood with agonists of TLR4, including monophosphoryl lipid A (MPLA) and glucopyranosyl lipid Adjuvant aqueous formulation (GLA-AF), as well as agonists of TLR7/8 (R848) and TLR9 (CpG). Both TLR4 agonists, MPLA and GLA-AF, induced greater concentrations of Th1 cytokines CXCL10, TNF and Interleukin (IL)-12p70 in infant and adult blood compared to newborn blood. All the tested TLRAs induced greater infant IFN-α2 production compared to newborn and adult blood. In contrast, CpG induced greater IFN-γ, IL-1ß, IL-4, IL-12p40, IL-10 and CXCL8 in newborn than in infant and adult blood. Overall, to the extent that these in vitro studies mirror responses in vivo, our study demonstrates distinct age-specific effects of TLRAs that may inform their development as candidate adjuvants for early life vaccines.

Missed Appointments: Factors Contributing to High No-Show Rates in an Urban Pediatrics Primary Care Clinic.

BACKGROUND: Missed appointments complicate primary care services. OBJECTIVE: To determine factors associated with missed pediatric appointments. DESIGN/METHODS: A convenience sample of 1537 patients who missed appointments were called and 386 (25%) families completed the 26-item survey. Those with high no-show rates were compared with the rest using χ(2) and Fisher's exact tests. Initial covariates with P < .2 were included in a multivariate logistic regression model. RESULTS: Common reasons for missing appointments were the following: forgot (27%), transportation problems (21%), and time off of work (14%). The high no-show group had more African Americans (P = .030) and older patients (P = .003). Higher no-show rates correlated with well child visits (P = .029) and perception of "excellent health" (P = .022). In the logistic regression model, well child appointments (odds ratio = 2.56) and increasing age in years (odds ratio = 1.11) were associated with higher no-show rates. CONCLUSIONS: Efforts to decrease no-show rates should target older patients and well child visits.

Improving immunization rates in a hospital-based primary care practice.

OBJECTIVE: We implemented a quality improvement initiative aimed at reaching a 95% immunization rate for patients aged 24 months. The setting was a hospital-based pediatric primary care practice in Boston, Massachusetts. We defined immunization as full receipt of the vaccine series as recommended by the Centers for Disease Control and Prevention. METHODS: The initiative was team-based and structured around 3 core interventions: systematic identification and capture of target patients, use of a patient-tracking registry, and patient outreach and care coordination. We measured monthly overall and modified immunization rates for patients aged 24 months. The modified rate excluded vaccine refusals and practice transfers. We plotted monthly overall and modified immunization rates on statistical process control charts to monitor progress and evaluate impact. RESULTS: We measured immunization rates for 3298 patients aged 24 months between January 2009 and December 2012. Patients were 48% (n = 1576) female, 77.3% (n = 2548) were African American or Hispanic, and 70.2% (n = 2015) were publicly insured. Using control charts, we established mean overall and modified immunization rates of 90% and 93%, respectively. After implementation, we observed an increase in the mean modified immunization rate to 95%. CONCLUSIONS: A quality improvement initiative enabled our pediatric practice to increase its modified immunization rate to 95% for children aged 24 months. We attribute the improvement to the incorporation of medical home elements including a multidisciplinary team, patient registry, and care coordination.

Overweight prevention in pediatric primary care: a needs assessment of an urban racial/ethnic minority population.

The authors studied the prevalence of overweight-related behaviors in an urban clinic population, parents' perceived willingness to change, and identified potential gaps in nutrition and physical activity promotion. A total of 324 parents of children aged 3 to 13 years were surveyed. Clinical heights and weights were used to calculate body mass index (BMI). Of the 324 children in the study, 55% were black and 28% were Hispanic. Approximately 151 (47%) children had a BMI > or =85th percentile, and overweight-related behaviors, such as TV viewing, were highly prevalent. Overall, parents reported a need for counseling to help their children eat healthier and be more active and seemed willing to make behavior changes in these areas. However, their willingness to change appeared lowest in areas that may improve their child's weight status such as decreasing sedentary time and portion sizes. Overweight prevention efforts in primary care should include strategies to help clinicians negotiate behavior change with families.

Paediatric experiences with work-hour limitations.

OBJECTIVES: To evaluate the perceived impact of work-hour limitations on paediatric residency training programmes and to determine the various strategies used to accommodate these restrictions. METHODS: A three-page pre-tested survey was administered to programme directors at the 2004 Association of Paediatric Programme Directors meeting. The impact of work-hours was evaluated with Likert-type questions and the methods used to meet work-hour requirements were compared between large programmes (>or=30 residents) and small programmes. RESULTS: Surveys were received from 53 programme directors. The majority responded that work-hour limitations negatively impacted inpatient continuity, time for education, schedule flexibility and attending staff satisfaction. Supervision by attending staff was the only aspect to significantly improve. Perceived resident satisfaction was neutral. To accommodate work-hour limitations, 64% of programmes increased clinical responsibility to existing non-resident staff, 36% hired more non-resident staff and 17% increased the number of residents. Only one programme hired additional non-clinical staff. Large programmes were more likely to use more total methods on the inpatient wards (P < 0.01) and in the intensive care units (P < 0.05) to accommodate work-hour limitations. CONCLUSIONS: Programme directors perceived a negative impact of work-hours on most aspects of training without a perceived difference in resident satisfaction. While a variety of methods are used to accommodate work-hour limitations, programmes are not widely utilizing non-clinical staff to alleviate clerical burdens.

Immunizations in children with special health care needs in a medical home model of care.

OBJECTIVE: To assess whether a model of care with augmented services within an existing medical home could improve immunization rates for Children with Special Health Care Needs (CSHCN) and to evaluate the influence of family and practice characteristics on these children's immunization rates. METHOD: Participants were six general pediatric practices, CSHCN, and their families. Enhancements were added to the medical home for CSHCN, including additional nurse practitioner time. A prospective survey and chart review of volunteer families was made in 1998 repeating the chart review in 2000. RESULTS: The immunization rates of CSHCN in the practices before the intervention was 83.4% for DTaP, polio and MMR and 80% if HIB and HepB were added. The intervention did not improve these already high rates. In regression models, children with late onset disorders were more likely NOT to be up-to-date (UTD) by 2 years of age (OR 3.29). CSHCN cared for by non-birth parents and CSHCN whose family incomes were greater than $40,000 per year were also more likely NOT to be UTD (OR 3.81, 2.75 respectively) as were patients of providers not satisfied with any aspect of serving patients (OR 3.29). CONCLUSIONS: In these practices already committed to a medical home model of care for CSHCN, immunization rates were at or above state and national averages before any intervention took place. Some groups of patients among these CSHCN were more likely to be delayed with immunizations, including those with late onset disorders and those with high family incomes.

Pediatric residents address critical child health issues in the community.

BACKGROUND: Practicing pediatricians are challenged to look beyond the clinical settings for local resources to address health threats to children and families. Pediatric residents' perceptions of the realities faced by families that they frequently see in the emergency room and outpatient or inpatient units are inadequate. OBJECTIVES: The Community Pediatrics Training Initiative (CPTI) is a national initiative within the American Academy of Pediatrics designed to prepare future pediatricians to serve families from traditionally underserved communities. The three unique components of this CPTI at Children's Hospital of Wisconsin (CHW) and the Medical College of Wisconsin (MCW) support child health improvement through (1) interactive discussions with families, (2) exposure to programs in community agencies, and (3) community initiatives. METHODS: Pediatric residents, along with community partners, have implemented more than 25 community initiatives in the areas of access to care, adolescent health, child advocacy, and environmental and public health. RESULTS: The immediate assessment from our CPTI curriculum demonstrates community experiences significantly promote their competencies in child advocacy and system-based practice while expanding their knowledge of the impact of poverty. In 2005, a survey of 84 postgraduates (75% response rate) assessed the effectiveness of community pediatrics training. All graduates self-reported that they are moderately to greatly prepared to respond to children's culture, age, gender, and disabilities. Of the 43 (68%) graduates who are currently in practice, 10 (23%) are working directly with underserved populations. CONCLUSIONS: The development of community initiatives by pediatric residents in partnership with community stakeholders greatly expanded residents' competencies in child health improvement.

Lessons from pediatrics residency program directors' experiences with work hour limitations in New York State.

PURPOSE: To evaluate the impact of residency work hour limitations on pediatrics residency programs in New York State, and to learn lessons that can be used nationally with the implementation of the Accreditation Council of Graduate Medical Education's similar rules. METHOD: A three-page questionnaire was mailed to all pediatrics residency program directors in New York. The questionnaire assessed methods used to accommodate the work hour limitations and perceptions of the limitations' effects. RESULTS: Twenty-one program directors responded (68%). Only large programs used night floats and night teams to meet work hour requirements. Programs of all sizes and in all settings used cross coverage and sent residents home immediately post call. About half of the programs hired additional nonresident staff, usually nurse practitioners, physician assistants, and/or attendings. The most frequently reported effects were decreases in the amount of time residents spent in inpatient settings, patient continuity in inpatient settings, flexibility of residents' scheduling, and increased logistical work needed to maintain continuity clinic. A summary of advice to other program directors was "be creative" and "be flexible." CONCLUSIONS: New York's pediatrics residency programs used a variety of mechanisms to meet work hour restrictions. Smaller programs had fewer methods available to them to meet such restrictions. Although the logistical work needed to maintain continuity clinic increased greatly, continuity and outpatient settings themselves were not greatly affected by work hour limitations. Inpatient settings were more affected and experienced much more in the way of change.

Herbs and other dietary supplements: healthcare professionals' knowledge, attitudes, and practices.

CONTEXT: Herbs and other dietary supplements (H/DS) are frequently used by the public. They have significant health implications, yet little is known about health professionals' knowledge, attitudes, or clinical practices related to H/DS. DESIGN: Cross-sectional survey of clinicians prior to participation in an Internet-based educational program on herbs and dietary supplements. PARTICIPANTS: The 537 participants included 111 physicians (MD), 30 advanced practice nurses (RN), 46 pharmacists (PharmD), and 350 dietitians (RD). In addition to demographic information, participants were asked about their knowledge, attitudes, and practices related to H/DS. RESULTS: Most participants were involved in direct patient care (85%), in practice or on faculty (84%), and from outside our local institutions (76%); 66% reported receiving professional education about H/DS in the past year. There were statistically significant differences between professional groups, with RDs scoring better than others, but even their average scores were less than 60% of possible. The average score on knowledge questions was 10/20; the average confidence score was 4 out of 10 possible, and the average communication score was 1.4 out of 4 possible. Most respondents knew the most common clinical uses of echinacea and St. John's wort, and felt confident that they knew more than their colleagues about H/DS. Key deficits were in knowledge about adverse effects, confidence in reporting side effects, routinely communicating with patients about H/DS, and recording H/DS information in the medical record. CONCLUSIONS: Despite significant interest and previous training in H/DS, these clinicians had substantial room for improvement in knowledge, attitudes, and clinical practices about H/DS. Educational interventions and institutional policies are needed to improve the quality of patient care regarding H/DS, and such interventions should be rigorously evaluated to ensure that continuous improvements occur.

Randomized trial of an internet curriculum on herbs and other dietary supplements for health care professionals.

PURPOSE: Herbs and dietary supplements are widely used, and there is an urgent need to educate health professionals about their risks and benefits. The Internet provides an innovative way of educating busy health professionals. This study assessed the impact of an Internet-based curriculum on health professionals' knowledge, confidence, and clinical practices related to herbs and dietary supplements. METHOD: The study was a randomized crossover trial involving physicians, pharmacists, advanced practice nurses, and dietitians. Participants were invited by e-mail and randomly assigned to immediate intervention versus waiting-list groups (n = 537). The curriculum lasted ten weeks and consisted of 20 case-based modules, each involving one multiple-choice or true-or-false question and its answer, links to reliable Internet sources of additional information, and a moderated listserv discussion group. Participants were surveyed about their knowledge, confidence, and communication related to herbs and dietary supplements on enrollment, after the immediate intervention group had completed the curriculum, and after the waiting-list group completed the curriculum. RESULTS: Baseline scores for knowledge, confidence, and communication were similar in the two groups. At the first follow-up, there was greater improvement in scores for all three areas in the immediate intervention group than in the waiting-list group (improvement for knowledge, 3.0 versus 1.4; confidence, 2.6 versus 0.6; communication, 0.21 versus -0.1, p <.01 for all comparisons). After all participants had received the curriculum, scores for both groups were significantly better than at baseline and similar to one another. CONCLUSION: Internet-based education about herbs and dietary supplements for diverse health professionals is feasible and results in significant and sustained improvements in knowledge, confidence, and communication practices.

Improving accuracy in a computerized immunization registry.

BACKGROUND: Boston's Department of Health and Hospitals established the Boston Immunization Information System (BIIS) as part of a city-wide effort to raise immunization rates. OBJECTIVES: Our objectives were 1) to assess the validity of data entry in one computerized immunization database within this system; 2) to identify the types of errors made in data entry; and 3) to assess the effectiveness of an intervention to improve the accuracy of information in this database. METHODS: Chart records were used as the gold standard in comparison with the computerized BIIS database. Children were considered up to date for immunizations if they had received 4 DTP, 3 polio, and 1 MMR by their second birthday. In March of 1995, billing records were used to identify all children born between March 1, 1992, and September 1, 1994, in one urban health center. These children were between 6 months and 36 months of age at the time. We compared the computer record with the chart, looking for and correcting errors such as incorrect dates and missing immunization data. An intervention was then begun, including a system for reviewing the accuracy of the computerized data at all well-child visits. In October of 1996, the chart review was repeated on children born between March 1, 1992, and September 1, 1995. This age range included all the children in the original review plus those born in the subsequent 12 months. Immunization rates before and after chart reviews were compared for children 24 to 36 months of age. RESULTS: We initially reviewed 737 of 739 charts (99%). In the follow-up time period, we examined 881 of 943 charts (93%). During the first review, 333 of the 563 (59%) records contained at least one error in data entry, compared with 116 of 646 (18%) in the second review (P <.0001). During the second review, we also examined the type of errors. Thirty-eight percent of all errors represented vaccines that had not been entered into the computer. Before the study period, analysis of the computerized immunization record showed an immunization up-to-date rate of 24%. This increased to 41% after the initial chart review and to 75% after the intervention period (P <.0001). CONCLUSION: Errors in data entry caused underestimates of immunization rates. Eliminating the errors increased immunization rates immediately through more accurate bookkeeping. Eighteen months later, immunization rates had continued to improve dramatically.