Economic review of immunization information systems to increase vaccination rates: a community guide systematic review.

CONTEXT: A recent systematic review found that use of an immunization information system (IIS) is an effective intervention to increase vaccination rates. The purpose of this review was to evaluate costs and benefits associated with implementing, operating, and participating with an IIS. The speed of technology change has had an effect on costs and benefits of IIS and is considered in this review. EVIDENCE ACQUISITION: An economic evaluation for IIS was conducted using methods developed for Community Guide systematic reviews. The literature search covered the period from January 1994 to March 2012 and identified 12 published articles and 2 government reports. EVIDENCE SYNTHESIS: Most studies involving cost data evaluated (1) system costs of building an IIS and (2) cost of exchanging immunization data; most economic benefits focused on administrative efficiency. CONCLUSIONS: A major challenge to evaluating a technology-based intervention is the evolution that comes with technology improvements and advancements. Although the cost and benefit data may be less applicable today due to changes in system technology, data exchange methods, availability of vendor support, system functionalities, and scope of IIS, it is likely that more up-to-date estimates and comprehensive estimates of benefits would support the findings of cost savings in this review. More research is needed to update and address limitations in the available evidence and to enable assessment of economic costs and benefits of present-day IIS.

Monitoring and reporting the US COVID-19 vaccination effort.

Immunizations are an important tool to reduce the burden of vaccine preventable diseases and improve population health.1 High-quality immunization data is essential to inform clinical and public health interventions and respond to outbreaks of vaccine-preventable diseases. To track COVID-19 vaccines and vaccinations, CDC established an integrated network that included vaccination provider systems, health information exchange systems, immunization information systems, pharmacy and dialysis systems, vaccine ordering systems, electronic health records, and tools to support mass vaccination clinics. All these systems reported data to CDC's COVID-19 response system (either directly or indirectly) where it was processed, analyzed, and disseminated. This unprecedented vaccine tracking effort provided essential information for public health officials that was used to monitor the COVID-19 response and guide decisions. This paper will describe systems, processes, and policies that enabled monitoring and reporting of COVID-19 vaccination efforts and share challenges and lessons learned for future public health emergency responses.

Current Challenges and Future Possibilities for Immunization Information Systems.

Immunizations have proven to be an important tool for public health and for reducing the impact of vaccine preventable diseases. To realize the maximum benefits of immunizations, a coordinated effort between public policy, health care providers and health systems is required to increase vaccination coverage and to ensure high-quality data are available to inform clinical and public health interventions. Immunization information systems (IIS) are confidential, population-based, computerized databases that record all immunization doses administered by participating providers to persons residing within a given geopolitical area. The key output of an IIS is high-quality data for use in targeting and monitoring immunization program activities and providing clinical decision support at the point of care. To be truly effective, IISs need to form a nationwide network and repository of immunization data. Since the early 2000s Centers for Disease Control and Prevention has made strides to help IIS move toward a nationwide network through efforts focused on improving infrastructure and functionality, such as the IIS Minimum Functional Standards, and the IIS Annual Report, a self-reported data collection of IIS progress toward achieving the functional standards. While these efforts have helped immunization programs achieve more functional standards, there is a need to shift focus from infrastructure and functionality improvements to high data quality through objective measurement of IIS performance and evaluating critical outcomes. Additionally, realizing the vision of a nationwide repository of high-quality immunization data requires tackling the many challenges that impact data quality and availability including those related to policy, data sharing, data use, aging IIS technology, sustainability, and participation in the IIS. This paper describes the current state of IIS in the United States, critical challenges impacting the quality of data in IIS, and potential components of a future IIS model to address these challenges.

Clinical Decision Support for Immunization Uptake and Use in Immunization Health Information Systems.

CONTEXT: In the United States, immunization recommendations and their associated schedules are developed by the Advisory Committee on Immunization Practices (ACIP). To assist with the translation process and better harmonize the outcomes of existing clinical decision support tools, the Centers for Disease Control and Prevention (CDC) created clinical decision support for immunization (CDSi) resources for each set of ACIP recommendations. These resources are continually updated and refined as new vaccine recommendations and clarifications become available and will be available to health information systems for a coronavirus disease 2019 (COVID-19) vaccine when one becomes available for use in the United States Objectives: To assess awareness of CDSi resources, whether CDSi resources were being used by immunization-related health information systems, and perceived impact of CDSi resources on stakeholders' work Design: Online surveys conducted from 2015-2019 including qualitative and quantitative questions Participants: The main and technical contact from each of the 64 CDC-funded immunization information system (IIS) awardees, IIS vendors, and electronic health record vendors Results: Awareness of at least one resource increased from 75% of respondents in 2015 to 100% in 2019. Use of at least one CDSi resource also increased from 47% in 2015 to 78% in 2019. About 80% or more of users of CDSi are somewhat or very highly satisfied with the resources and report a somewhat or very positive impact from using them Conclusion: As awareness and use of CDSi resources increases, the likelihood that patients receive recommended immunizations at the right time will also increase. Rapid and precise integration of vaccine recommendations into health information systems will be particularly important when a COVID-19 vaccine becomes available to help facilitate vaccine implementation.

An Internet-based migraine headache diary: issues in Internet-based research.

OBJECTIVE: The primary purpose of this study was to explore the feasibility and acceptability of using an Internet-based headache diary to obtain acceptable completion rates of daily diaries. BACKGROUND: Migraine sufferers often perceive that headaches are unpredictable, but 70% have prodromal warning symptoms that may be identified via daily headache diaries. Although diaries are widely used for tracking headaches, Internet-based diaries have not been used previously. METHODS: A conventional headache diary was formatted for the Internet to collect daily headache data over 4 months using a time-series design.Women between 18 and 55 years who were not pregnant or postmenopausal, and whose headaches met migraine criteria, were recruited primarily via the Internet, completed online consent forms, and were screened via telephone. They completed health history questionnaires and daily diary pages containing scales and open-ended questions,which were saved to a database. Diaries were reviewed and participants were contacted weekly. Completion dates were tracked electronically. Follow-up interviews addressed perceptions about study experiences, and participants received feedback about headache patterns. RESULTS: The majority of participants were recruited from discussion boards and free classified web sites. Of the 101 participants enrolled, 24 withdrew prior to completing 4 months of diary entries. Participants (n = 77) had a mean age of 37.5(7.5) years and were primarily white (82%) and well-educated (93%). They lived in 21 US states, and one in the UK. The majority (68%) completed at least 50% of their diary pages within 24 hours; 75% of all pages were completed within 2 days. At least 64 (83%) kept notes or printed pages when they lacked Internet access. In a follow-up survey (n = 67), 87% would have been willing to continue the diary for another 2 months; 69% had not previously participated in any research. Participants also reported that the study helped them better understand their headache patterns, that the study was a major commitment but worthwhile, and that they felt they had helped others by participating. CONCLUSION: The Internet-based headache diary is a feasible, acceptable data collection tool that can access geographically diverse populations who have not previously participated in research studies. Use of an Internet-based approach was found to be feasible for recruitment and retention of such diverse populations.

Frequency and cost of live vaccines administered too soon after prior live vaccine in children aged 12 months through 6 years, 2014-2017.

OBJECTIVE: To identify number of children who received live vaccines outside recommended intervals between doses and calculate corrective revaccination costs. METHODS: We analyzed >1.6 million vaccination records for children aged 12 months through 6 years from six immunization information system (IIS) Sentinel Sites from 2014-15 when live attenuated influenza vaccine (LAIV, FluMist® Quadrivalent) was recommended for use, and from 2016-17, when not recommended for use. Depending on the vaccine, insufficient intervals between live vaccine doses are less than 24 or 28 days from a preceding live vaccine dose. Private and public purchase costs of vaccines were used to determine revaccination costs of live vaccine doses administered during the live vaccine conflict interval. Measles, mumps, rubella (MMR), varicella, combined MMRV, and LAIV were live vaccines evaluated in this study. RESULTS: Among 946,659 children who received at least one live vaccine dose from 2014-15, 4,873 (0.5%) received at least one dose too soon after a prior live vaccine (revaccination cost, $786,413) with a median conflict interval of 16 days. Among 704,591 children who received at least one live vaccine dose from 2016-17, 1,001 (0.1%) received at least one dose too soon after a prior live vaccine (revaccination cost, $181,565) with a median conflict interval of 14 days. The live vaccine most frequently administered outside of the recommended intervals was LAIV from 2014-15, and varicella from 2016-17. CONCLUSIONS: Live vaccine interval errors were rare (0.5%), indicating an adherence to recommendations. If all invalid doses were corrected by revaccination over the two time periods, the cost within the IIS Sentinel Sites would be nearly one million dollars. Provider awareness about live vaccine conflicts, especially with LAIV, could prevent errors, and utilization of clinical decision support functionality within IISs and Electronic Health Record Systems can facilitate better vaccination practices.

Using Internet discussion boards as virtual focus groups.

While the potential of Internet-based qualitative research methods is substantial, such methods are not without their problems. Some of these methodological challenges are unique to the medium, while others are similar to those of more traditional qualitative methods. This article presents some of these methodological challenges, and explores some of the issues involved in using on-line discussion boards as virtual focus groups in a study of perimenopausal women with migraines. Design of the study and its advantages and disadvantages are discussed, including the role of the moderator. Some of the problems encountered included potential for misunderstandings due to limits of written communications, and difficulty encouraging participation.

Measurement issues related to data collection on the World Wide Web.

As the World Wide Web has become more prominent as a mode of communication, it has opened up new possibilities for research data collection. This article identifies measurement issues that occur with Internet data collection that are relevant to qualitative and quantitative research approaches as they occurred in a triangulated Internet study of perimenopausal women with migraine headaches. Issues associated with quantitative data collection over the Internet include (a) selecting and designing Internet data collection protocols that adequately address study aims while also taking advantage of the Internet, (b) ensuring the reliability and validity of Internet data collected, (c) adapting quantitative paper-and-pencil data collection protocols for the Internet, (d) making Internet data collection practical for respondents and researchers, and (e) ensuring the quality of quantitative data collected. Qualitative data collection over the Internet needs to remain true to the philosophical stance of the qualitative approach selected. Researcher expertise in qualitative data collection must be combined with expertise in computer technology and information services if data are to be of ultimate quality The advantages and limitations of collecting qualitative data in real time or at a later time are explored, as well as approaches to enhance qualitative data collection over the Internet. It was concluded that like any research approach or method, Internet data collection requires considerable creativity, expertise, and planning to take advantage of the technology for the collection of reliable and valid research data.

A novel system for collecting longitudinal self-reported dietary consumption information: the internet data logger (iDL).

As diet has been considered as the major exposure pathway for toxic chemicals, such as heavy metals and pesticides, we have developed a web-based questionnaire system, the internet data logger (iDL), in order to facilitate the collection of detailed self-recording of standardized daily dietary consumption. An iDL prototype optimized for hand-held devices was tested in the on-going Children's Pesticide Exposure Study (CPES) in Atlanta, GA in which 15 consecutive days of dietary consumption information were recorded by 11 children or their parents and transmitted to a server via broadband wireless internet connection. Participants spent an average of 6.2 min in day 1 to enter a meal into iDL and approximately 4 min per meal for the remaining 14 study days. Approximately 75% of 446 total meals were entered into iDL on the date of consumption. A within-person comparison revealed that many food items recorded on paper diary were later classified as "unknown" according to iDL standards due to the lack of detailed information provided by the participants. The development of iDL represents a novel and pioneering tool, which integrates dietary data collection, transmission, and management in a real-time and automated manner. iDL provides the needed flexibility and mobility for research focused on collecting not only dietary consumption data but also other time-sensitive behavior data. This study has demonstrated that iDL can be deployed to collect long-term dietary consumption information in a population-based study, though future modifications/enhancements will undoubtedly improve its usability and application.

Online Data Collection in Women's Health Research: A Study of Perimenopausal Women with Migraines.

An estimated 17 to 18 percent of all women, and six percent of men, experience migraines. Hormonal shifts may cause migraines to recur, worsen, or even begin during the perimenopause and are a significant cause of discomfort and disability. However, very little research has explored the experience of migraines in this population. The purpose of this study was to describe the experiences of perimenopausal women with migraines, via online questionnaires and discussion boards, and to evaluate the feasibility of collecting women's health data via the Internet. In an earlier study, we found that midlife women had difficulty attending focus groups due to other time commitments. This study was designed to increase accessibility to the research via the Internet. Of the 43 women recruited into the study, 21 were also interviewed in "real-time" qualitative interviews; all received passwords to complete online questionnaires and participate in three- to four-week discussion boards on the study Web site. Quantitative data were imported into SPSS; narrative qualitative data from discussion boards were transferred to a software package for analysis. Online questionnaires and discussion boards were found to be feasible methods for data collection for this population. Qualitative data analysis revealed themes related to women's efforts to predict and control their headaches, the relationship of headaches to women's menses and menopausal symptoms, and the effects of migraines on their lives. In this paper we describe the process of using the Internet, feminist issues related to this innovative methodology, and also discuss the results of a major study theme, the experience of headaches in relationship to the menstrual cycle.