Electronic immunization data collection systems: application of an evaluation framework.

BACKGROUND: Evaluating the features and performance of health information systems can serve to strengthen the systems themselves as well as to guide other organizations in the process of designing and implementing surveillance tools. We adapted an evaluation framework in order to assess electronic immunization data collection systems, and applied it in two Ontario public health units. METHODS: The Centers for Disease Control and Prevention's Guidelines for Evaluating Public Health Surveillance Systems are broad in nature and serve as an organizational tool to guide the development of comprehensive evaluation materials. Based on these Guidelines, and informed by other evaluation resources and input from stakeholders in the public health community, we applied an evaluation framework to two examples of immunization data collection and examined several system attributes: simplicity, flexibility, data quality, timeliness, and acceptability. Data collection approaches included key informant interviews, logic and completeness assessments, client surveys, and on-site observations. RESULTS: Both evaluated systems allow high-quality immunization data to be collected, analyzed, and applied in a rapid fashion. However, neither system is currently able to link to other providers' immunization data or provincial data sources, limiting the comprehensiveness of coverage assessments. We recommended that both organizations explore possibilities for external data linkage and collaborate with other jurisdictions to promote a provincial immunization repository or data sharing platform. CONCLUSIONS: Electronic systems such as the ones described in this paper allow immunization data to be collected, analyzed, and applied in a rapid fashion, and represent the infostructure required to establish a population-based immunization registry, critical for comprehensively assessing vaccine coverage.

Influenza vaccination coverage across ethnic groups in Canada.

BACKGROUND: The success of influenza vaccination campaigns may be suboptimal if subgroups of the population face unique barriers or have misconceptions about vaccination. We conducted a national study to estimate influenza vaccine coverage across 12 ethnic groups in Canada to assess the presence of ethnic disparities. METHODS: We pooled responses to the Canadian Community Health Survey between 2003 and 2009 (n = 437 488). We estimated ethnicity-specific self-reported influenza vaccine coverage for the overall population, for people aged 65 years and older, and for people aged 12-64 years with and without chronic conditions. We used weighted logistic regression models to examine the association between ethnicity and influenza vaccination, adjusting for sociodemographic factors and health status. RESULTS: Influenza vaccination coverage ranged from 25% to 41% across ethnic groups. After adjusting for sociodemographic factors and health status for people aged 12 years and older, all ethnic groups were more likely to have received a vaccination against influenza than people who self-identified as white, with the exception of those who self-identified as black (odds ratio [OR] 1.01, 95% confidence interval [CI] 0.88-1.15). Compared with white Canadians, Canadians of Filipino (OR 2.00, 95% CI 1.67-2.40) and Southeast Asian (OR 1.66, 95% CI 1.36-2.03) descent had the greatest likelihood of having received vaccination against influenza. INTERPRETATION: Influenza vaccine coverage in Canada varies by ethnicity. Black and white Canadians have the lowest uptake of influenza vaccine of the ethnic groups represented in our study. Further research is needed to understand the facilitators, barriers and misconceptions relating to vaccination that exist across ethnic groups, and to identify promotional strategies that may improve uptake among black and white Canadians.

Barriers to the use of reminder/recall interventions for immunizations: a systematic review.

BACKGROUND: Although many studies have demonstrated the benefits of reminder/recall (RR) measures to address patient under-immunization and improve immunization coverage, they are not widely implemented by healthcare providers. We identified providers' perceived barriers to their use from existing literature. METHODS: We conducted a systematic review of relevant articles published in English between January 1990 and July 2011 that examined the perceptions of healthcare providers regarding barriers to tracking patient immunization history and implementing RR interventions. We searched MEDLINE, PubMed, EMBASE, Cumulative Index to Nursing and Allied Health Literature, Academic Search Premier, and PsychINFO. Additional strategies included hand-searching the references of pertinent articles and related reviews, and searching keywords in Google Scholar and Google. RESULTS: Ten articles were included; all described populations in the United States, and examined perceptions of family physicians, pediatricians, and other immunization staff. All articles were of moderate-high methodological quality; the majority (n=7) employed survey methodology. The most frequently described barriers involved the perceived human and financial resources associated with implementing an RR intervention, as well as low confidence in the accuracy of patient immunization records, given the lack of data sharing between multiple immunization providers. Changes to staff workflow, lack of appropriate electronic patient-tracking functionalities, and uncertainty regarding the success of RR interventions were also viewed as barriers to their adoption. CONCLUSIONS: Although transitioning to electronic immunization records and registries should facilitate the implementation of RR interventions, numerous perceived barriers must still be overcome before the full benefits of these methods can be realized.

A cost comparison of electronic and hybrid data collection systems in Ontario during pandemic and seasonal influenza vaccination campaigns.

BACKGROUND: During the pandemic (H1N1) 2009 influenza vaccination campaign, health regions in Canada collected client-level immunization data using fully electronic or hybrid systems, with the latter comprising both electronic and paper-based elements. The objective of our evaluation was to compare projected five-year costs associated with implementing these systems in Ontario public health units (PHUs) during pandemic and seasonal influenza vaccination campaigns. METHODS: Six PHUs provided equipment and staffing costs during the pandemic (H1N1) 2009 influenza vaccination campaign and staffing algorithms for seasonal campaigns. We standardized resources to population sizes 100,000, 500,000 and 1,000,000, assuming equipment lifetime of five years and public health vaccine administration rates of 18% and 2.5% for H1N1 and seasonal campaigns, respectively. Two scenarios were considered: Year 1 pandemic and Year 1 seasonal campaigns, each followed by four regular influenza seasons. Costs were discounted at 5%. RESULTS: Assuming a Year 1 pandemic, the five-year costs per capita for the electronic system decrease as PHU population size increases, becoming increasingly less costly than hybrid systems ($4.33 vs. $4.34 [100,000], $4.17 vs. $4.34 [500,000], $4.12 vs. $4.34 [1,000, 000]). The same trend is observed for the scenario reflecting five seasonal campaigns, with the electronic system being less expensive per capita than the hybrid system for all population sizes ($1.93 vs. $1.95 [100,000], $1.91 vs. $1.94 [500,000], $1.87 vs. $1.94 [1,000, 000]). Sensitivity analyses identified factors related to nurse hours as affecting the direction and magnitude of the results. CONCLUSIONS: Five-year cost projections for electronic systems were comparable or less expensive than for hybrid systems, at all PHU population sizes. An intangible benefit of the electronic system is having data rapidly available for reporting.

Approaches to immunization data collection employed across Canada during the pandemic (H1N1) 2009 influenza vaccination campaign.

OBJECTIVES: A critical component of the 2009 H1N1 vaccination campaign was the collection of immunization data at the point of care. To meet reporting requirements and to ensure timely availability of coverage information, many jurisdictions across Canada employed new or modified approaches to vaccine data collection. The objective of this study was to observe and characterize the range of influenza immunization data collection approaches used across Canada. METHODS: As part of a multi-stage observational study, the research team visited immunization clinics at which tasks related to data collection and management were observed. Tasks included registration, medical history collection and review, vaccine record-keeping, proof of vaccination preparation, and data entry. Field notes were analyzed in order to understand the data collection mechanisms that comprised each information system as a whole. RESULTS: Data collection mechanisms were grouped into two categories: electronic systems (9/38), in which all data were captured on computer; and hybrid systems (29/38), comprised of computerized and paper-based data collection tasks. Observed systems included stand-alone databases, immunization registries, and electronic health records. Organizations incorporated magnetic card reader technology, telephone registration, and pre-populated fields into data collection approaches. Electronic systems captured a greater number of data elements. CONCLUSION: Canadian jurisdictions employed a range of data collection approaches during the H1N1 vaccination campaign. System characteristics can have important implications for on-site efficiency and organization as well as program planning and evaluation. The systems observed have been described in detail to allow vaccine providers and planners to learn from what has been done elsewhere.

Perceptions of frontline staff regarding data collection methodologies used during the 2009 A H1N1 influenza immunization campaign in Canada.

BACKGROUND: During the 2009 H1N1 immunization campaign, electronic and hybrid (comprising both electronic and paper components) systems were employed to collect client-level vaccination data in clinics across Canada. Because different systems were used across the country, the 2009 immunization campaign offered an opportunity to study the usability of the various data collection methods. METHODS: A convenience sample of clinic staff working in public health agencies and hospitals in 9 provinces/territories across Canada completed a questionnaire in which they indicated their level of agreement with seven statements regarding the usability of the data collection system employed at their vaccination clinic. Questions included overall ease of use, effectiveness of the method utilized, efficiency at completing tasks, comfort using the method, ability to recover from mistakes, ease of learning the method and overall satisfaction with the method. A 5-point Likert-type scale was used to measure responses. RESULTS: Most respondents (96%) were employed in sites run by public health. Respondents included 186 nurses and 114 administrative staff, among whom 90% and 47%, respectively, used a paper-based method for data collection. Approximately half the respondents had a year or less of experience with immunization-related tasks during seasonal influenza campaigns. Over 90% of all frontline staff found their data collection method easy to use, perceived it to be effective in helping them complete their tasks, felt quick and comfortable using the method, and found the method easy to learn, regardless of whether a hybrid or electronic system was used. CONCLUSIONS: This study demonstrates that there may be a greater willingness of frontline immunization staff to adapt to new technologies than previously perceived by decision-makers. The public health community should recognize that usability may not be a barrier to implementing electronic methods for collecting individual-level immunization data.

Perceptions of immunization information systems for collecting pandemic H1N1 immunization data within Canada's public health community: a qualitative study.

BACKGROUND: Immunization information systems (IISs) are electronic registries used to monitor individual vaccination status and assess vaccine coverage. IISs are currently not widely used across Canada, where health jurisdictions employ a range of approaches to capture influenza immunization information. Conducted in advance of the 2009 H1N1 vaccination campaign, the objectives of this study were to understand the perceived value of individual-level data and IISs for influenza control, identify ideal system functions, and explore barriers to implementation. METHODS: In July and August 2009, semi-structured interviews were conducted with key informants engaged in vaccine delivery and/or pandemic planning at regional, provincial/territorial and federal levels across Canada. Key informants were recruited using a combination of convenience and snowball sampling methodologies. Qualitative analysis was used to extract themes from interview content. RESULTS: Patient management, assessment of vaccine coverage, and evaluation of safety and effectiveness were identified as public health priorities that would be achieved in a more timely manner, and with greater accuracy, through the use of an IIS. Features described as ideal included system flexibility, rapid data entry, and universality. Financial and human resource constraints as well as coordination between immunization providers were expressed as barriers to implementation. CONCLUSIONS: IISs were perceived as valuable by key informants for strengthening management capacity and improving evaluation of both seasonal and pandemic influenza vaccination campaigns. However, certain implementation restrictions may need to be overcome for these benefits to be achieved.

Assessing Local Risk of Rifampicin-Resistant Tuberculosis in KwaZulu-Natal, South Africa Using Lot Quality Assurance Sampling.

BACKGROUND: KwaZulu-Natal (KZN) has the highest burden of notified multidrug-resistant tuberculosis (MDR TB) and extensively drug-resistant (XDR) TB cases in South Africa. A better understanding of spatial heterogeneity in the risk of drug-resistance may help to prioritize local responses. METHODS: Between July 2012 and June 2013, we conducted a two-way Lot Quality Assurance Sampling (LQAS) study to classify the burden of rifampicin (RIF)-resistant TB among incident TB cases notified within the catchment areas of seven laboratories in two northern and one southern district of KZN. Decision rules for classification of areas as having either a high- or low-risk of RIF resistant TB (based on proportion of RIF resistance among all TB cases) were based on consultation with local policy makers. RESULTS: We classified five areas as high-risk and two as low-risk. High-risk areas were identified in both Southern and Northern districts, with the greatest proportion of RIF resistance observed in the northernmost area, the Manguzi community situated on the Mozambique border. CONCLUSION: Our study revealed heterogeneity in the risk of RIF resistant disease among incident TB cases in KZN. This study demonstrates the potential for LQAS to detect geographic heterogeneity in areas where access to drug susceptibility testing is limited.

Assessing the utility of Xpert(®) MTB/RIF as a screening tool for patients admitted to medical wards in South Africa.

Many hospital inpatients in South Africa have undiagnosed active and drug-resistant tuberculosis (TB). Early detection of TB is essential to inform immediate infection control actions to minimize transmission risk. We assessed the utility of Xpert(®) MTB/RIF (GeneXpert) as a screening tool for medical admissions at a large public hospital in South Africa. Consecutive adult patients admitted to medical wards between March-June 2013 were enrolled; sputum specimens were collected and tested by GeneXpert, smear microscopy, and culture. Chest X-rays (CXRs) were conducted as standard care for all patients admitted. We evaluated the proportion of patients identified with TB disease through each diagnostic method. Among enrolled patients whose medical charts were available for review post-discharge, 61 (27%) were diagnosed with TB; 34 (56% of diagnosed TB cases) were GeneXpert positive. When patients in whom TB was identified by other means were excluded, GeneXpert yielded only four additional TB cases. However, GeneXpert identified rifampicin-resistant TB in one patient, who was initially diagnosed based on CXR. The utility of GeneXpert for TB screening was limited in an institution where CXR is conducted routinely and which serves a population in which TB and TB/HIV co-infection are highly prevalent, but it allowed for rapid detection of rifampicin resistance.

Health care worker influenza immunization rates: the missing pieces of the puzzle.

BACKGROUND: Immunization rates are used to assess the level of protection against influenza, but limited data exist on how such rates are measured in health care organizations. We conducted key informant interviews with campaign planners to learn about processes for collecting immunization data, including barriers and facilitating factors for measuring and reporting rates. METHODS: We conducted telephone interviews with 23 influenza immunization program planners across Canada working in 7 acute care hospitals, 6 continuing care facilities, and 8 public health organizations in 2012. We used content analysis to examine the interview data. RESULTS: The methods used to collect immunization data varied by the size and type of health care organization. Immunization data from different personnel groups were included in immunization rate calculations depending on the local public health reporting requirements and the organization's size. Challenges associated with collecting immunization data and calculating rates included lack of resources for identifying personnel immunized off-site, tracking personnel who declined immunization, identifying non-payroll staff, and interpreting unclear public health reporting requirements. CONCLUSION: Support from other vaccine providers, public health, employers, and professional and external bodies is needed to provide the necessary information and resources to calculate accurate and complete rates. Further work is needed to refine and standardize the collection of HCW influenza immunization data so that it may be used for surveillance and quality assessment purposes.

Measuring influenza immunization coverage among health care workers in acute care hospitals and continuing care organizations in Canada.

BACKGROUND: Immunizing health care workers against influenza is important for preventing and reducing disease transmission in health care environments. We describe the ability of Canadian health care organizations to measure influenza immunization coverage among health care workers and identify factors associated with comprehensive influenza immunization measurement. METHODS: A Web-based survey was distributed to influenza immunization campaign planners responsible for delivering the 2010-2011 influenza vaccine to health care workers working in acute care hospitals or long-term continuing care organizations. The primary outcome was the ability to comprehensively measure influenza immunization coverage. RESULTS: Of the 1,127 health care organizations approached, 721 (64%) responded. Ninety-one percent had incomplete immunization coverage measurement; 7% could not measure coverage among any personnel. After multivariable adjustment, organizations with a written influenza immunization implementation plan (odds ratio, 2.0; 95% confidence interval, 1.1-3.5) or a policy or procedure describing how to calculate or report immunization rates (odds ratio, 2.1; 95% confidence interval, 1.2-3.9) were more likely to have comprehensive measurement of influenza immunization coverage than organizations without these practices. CONCLUSION: Most organizations demonstrated incomplete measurement of influenza immunization among health care workers. Given the use of influenza immunization coverage as a measure of quality of care, further work is needed to develop a standardized approach to improve its measurement.

Incorporating scannable forms into immunization data collection processes: a mixed-methods study.

INTRODUCTION: Individual-level immunization data captured electronically can facilitate evidence-based decision-making and planning. Populating individual-level records through manual data entry is time-consuming. An alternative is to use scannable forms, completed at the point of vaccination and subsequently scanned and exported to a database or registry. To explore the suitability of this approach for collecting immunization data, we conducted a feasibility study in two settings in Ontario, Canada. METHODS AND FINDINGS: Prior to the 2011-2012 influenza vaccination campaign, we developed a scannable form template and a corresponding database that captured required demographic and clinical data elements. We examined efficiency, data quality, and usability through time observations, record audits, staff interviews, and client surveys. The mean time required to scan and verify forms (62.3 s) was significantly shorter than manual data entry (69.5 s) in one organization, whereas there was no difference (36.6 s vs. 35.4 s) in a second organization. Record audits revealed no differences in data quality between records populated by scanning versus manual data entry. Data processing personnel and immunized clients found the processes involved to be straightforward, while nurses and managers had mixed perceptions regarding the ease and merit of using scannable forms. Printing quality and other factors rendered some forms unscannable, necessitating manual entry. CONCLUSIONS: Scannable forms can facilitate efficient data entry, but certain features of the forms, as well as the workflow and infrastructure into which they are incorporated, should be evaluated and adapted if scannable forms are to be a meaningful alternative to manual data entry.

Exploring the feasibility of integrating barcode scanning technology into vaccine inventory recording in seasonal influenza vaccination clinics.

BACKGROUND: In response to the need for improved quality of vaccine inventory and client immunization records, barcodes containing a unique identifier and lot number will be placed on all vaccine vials in Canada. We conducted feasibility studies to examine integration of barcode scanning into inventory recording workflow for mass immunization clinics. METHODS: During the 2010-2011 seasonal influenza vaccination campaign, Ontario public health units (PHUs) using an electronic immunization system were randomized to record clinic inventory data (including vaccine lot number and expiry date) through: (i) barcode scanning of vials; or (ii) drop-down menus. A third group of PHUs recording vaccine inventory on paper served as an observation arm. We visited a sample of clinics within each PHU to assess barcode readability, method efficiency and data quality. Clinic staff completed a survey examining method perceptions. RESULTS: We observed 20 clinics using barcode scanning to record inventory data (eight PHUs), 20 using drop-down menus (eight PHUs), and 21 using paper forms (five PHUs). Mean time spent recording data per vial was 4.3s using barcode scanners with 1.3 scan attempts per vial, 0.5s using drop-down menus, and 1.7s using paper. Few errors were observed. Sixty-four perception surveys were completed by inventory staff; barcode scanning users indicated fairly strong overall satisfaction with the method (74%), and the majority agreed that barcode scanning improved client safety (84%) and inventory record accuracy (77%). However, 38% of barcode scanning users felt that individually scanning vials took longer than the other approaches and 26% indicated that this increased time would discourage them from adopting the method. CONCLUSIONS: Our study demonstrated good readability of barcodes but scanning individual vials for high-volume clinics was time-consuming; modifying the process will improve feasibility to facilitate adoption in Canada, while serving as an example for other countries considering this technology.

Time and motion study to compare electronic and hybrid data collection systems during the pandemic (H1N1) 2009 influenza vaccination campaign.

During the pandemic (H1N1) 2009 vaccination campaign, vaccine providers collected immunization data using hybrid (paper-based and electronic methods) and electronic data systems. We measured staff time in seconds spent on data collection tasks to compare system efficiencies. The sample consisted of 38 organizations across nine Canadian provinces/territories. The total mean data collection times per client were 104 s (electronic system), 143 s (hybrid system with electronic registration) and 172 s (hybrid system with paper registration). Electronic registration and record keeping were faster than paper-based methods; these findings should be used to improve data collection for future influenza seasons.