Decision Support for Signal Assessment of Large Case Series in Pharmacovigilance.

In pharmacovigilance, signal assessment of a medicinal product and adverse event can involve reviewing prohibitively large numbers of case reports. A prototype of a decision support tool guided by a needs assessment was developed to help manual review of many reports. In a preliminary qualitative evaluation, users said the tool was easy to use, improved efficiency and provided new insights.

E-Learning in Pharmacovigilance: An Evaluation of Microlearning-Based Modules Developed by Uppsala Monitoring Centre.

INTRODUCTION: An increasing global need for pharmacovigilance training cannot be met with classroom courses alone. Several e-learning modules have been developed by Uppsala Monitoring Centre (UMC). With distance learners and technological challenges such as poor internet bandwidth to be considered, UMC opted for the microlearning approach based on small learning units connected to specific learning objectives. The aim of this study was to evaluate how this e-learning course was received. METHODS: The course was evaluated through usage data and the results of two user surveys, one for modules 1-4, signal detection and causality assessment, and the other for module 5, statistical reasoning and algorithms in pharmacovigilance. The evaluation model used was based on the Unified Theory of Acceptance and Use of Technology (UTAUT). A questionnaire was developed, divided into demographic profile, performance expectancy, effort expectancy, educational compatibility and behavioural intention. The two surveys were disseminated to 2067 learners for modules 1-4 and 1685 learners for module 5. RESULTS: Learners from 137 countries participated, predominantly from industry (36.6%), national pharmacovigilance centres (22.6%) and academia (16.3%). The overall satisfaction level was very high for all modules, with over 90% of the learners rating it as either 'excellent' or 'good'. The majority were satisfied with the learning platform, the course content and the lesson duration. Most learners thought they would be able to apply the knowledge in practice. Almost 100% of the learners would recommend the modules to others and would also study future modules. Suggested improvements were an interactive forum, more practical examples in the lessons and practical exercises. CONCLUSION: This e-learning course in pharmacovigilance based on microlearning was well received with a global coverage among relevant professional disciplines.

A Feasibility Study of Drug-Drug Interaction Signal Detection in Regular Pharmacovigilance.

INTRODUCTION: Adverse drug reactions related to drug-drug interactions cause harm to patients. There is a body of research on signal detection for drug interactions in collections of individual case reports, but limited use in regular pharmacovigilance. OBJECTIVE: The aim of this study was to evaluate the feasibility of signal detection of drug-drug interactions in collections of individual case reports of suspected adverse drug reactions. METHODS: This study was conducted in VigiBase, the WHO global database of individual case safety reports. The data lock point was 31 August 2016, which provided 13.6 million reports for analysis after deduplication. Statistical signal detection was performed using a previously developed predictive model for possible drug interactions. The model accounts for an interaction disproportionality measure, expressed suspicion of an interaction by the reporter, potential for interaction through cytochrome P450 activity of drugs, and reported information indicative of unexpected therapeutic response or altered therapeutic effect. Triage filters focused the preliminary signal assessment on combinations relating to serious adverse events with case series of no more than 30 reports from at least two countries, with at least one report during the previous 2 years. Additional filters sought to eliminate already known drug interactions through text mining of standard literature sources. Preliminary signal assessment was performed by a multidisciplinary group of pharmacovigilance professionals from Uppsala Monitoring Centre and collaborating organizations, whereas in-depth signal assessment was performed by experienced pharmacovigilance assessors. RESULTS: We performed preliminary signal assessment for 407 unique drug pairs. Of these, 157 drug pairs were considered already known to interact, whereas 232 were closed after preliminary assessment for other reasons. Ten drug pairs were subjected to in-depth signal assessment and an additional eight were decided to be kept under review awaiting additional reports. The triage filters had a major impact in focusing our preliminary signal assessment on just 14% of the statistical signals generated by the predictive model for drug interactions. In-depth assessment led to three signals communicated with the broader pharmacovigilance community, six closed signals and one to be kept under review. CONCLUSION: This study shows that signals of adverse drug interactions can be detected through broad statistical screening of individual case reports. It further shows that signal assessment related to possible drug interactions requires more detailed information on the temporal relationship between different drugs and the adverse event. Future research may consider whether interaction signal detection should be performed not for individual adverse event terms but for pairs of drugs across a spectrum of adverse events.

Prospective Evaluation of Adverse Event Recognition Systems in Twitter: Results from the Web-RADR Project.

INTRODUCTION: A large number of studies on systems to detect and sometimes normalize adverse events (AEs) in social media have been published, but evidence of their practical utility is scarce. This raises the question of the transferability of such systems to new settings. OBJECTIVES: The aims of this study were to develop an AE recognition system, prospectively evaluate its performance on an external benchmark dataset and identify potential factors influencing the transferability of AE recognition systems. METHODS: A pipeline based on dictionary lookups and logistic regression classifiers was developed using a proprietary dataset of 196,533 Tweets manually annotated for AE relations and prospectively evaluated the system on the publicly available WEB-RADR reference dataset, exploring different aspects affecting transferability. RESULTS: Our system achieved 0.53 precision, 0.52 recall and 0.52 F1-score on the development test set; however, when applied to the WEB-RADR reference dataset, system performance dropped to 0.38 precision, 0.20 recall and 0.26 F1-score. Similarly, a previously published method aiming at automatically detecting adverse event posts reported 0.5 precision, 0.92 recall and 0.65 F1-score on thus another dataset, while performance on the WEB-RADR reference dataset was reduced to 0.37 precision, 0.63 recall and 0.46 F1-score. We identified four potential factors leading to poor transferability: overfitting, selection bias, label bias and prevalence. CONCLUSION: We warn the community about a potentially large discrepancy between the expected performance of automated AE recognition systems based on published results and the actual observed performance on independent data. This study highlights the difficulty of implementing an all-purpose system for automatic adverse event recognition in Twitter, which could explain the lack of such systems in practical pharmacovigilance settings. Our recommendation is to use benchmark independent datasets, such as the WEB-RADR reference, to investigate the transferability of the adverse event recognition systems and ultimately enforce rigorous comparisons across studies on the task.

Recommendations for the Use of Social Media in Pharmacovigilance: Lessons from IMI WEB-RADR.

Over a period of 3 years, the European Union's Innovative Medicines Initiative WEB-RADR project has explored the value of social media (i.e., information exchanged through the internet, typically via online social networks) for identifying adverse events as well as for safety signal detection. Many patients and clinicians have taken to social media to discuss their positive and negative experiences of medications, creating a source of publicly available information that has the potential to provide insights into medicinal product safety concerns. The WEB-RADR project has developed a collaborative English language workspace for visualising and analysing social media data for a number of medicinal products. Further, novel text and data mining methods for social media analysis have been developed and evaluated. From this original research, several recommendations are presented with supporting rationale and consideration of the limitations. Recommendations for further research that extend beyond the scope of the current project are also presented.

Boosting Quality Registries with Clinical Decision Support Functionality*. User Acceptance of a Prototype Applied to HIV/TB Drug Therapy.

BACKGROUND: The care of HIV-related tuberculosis (HIV/TB) is complex and challenging. Clinical decision support (CDS) systems can contribute to improve quality of care, but more knowledge is needed on factors determining user acceptance of CDS. OBJECTIVES: To analyze physicians' and nurses' acceptance of a CDS prototype for evidence-based drug therapy recommendations for HIV/TB treatment. METHODS: Physicians and nurses were involved in designing a CDS prototype intended for future integration with the Swedish national HIV quality registry. Focus group evaluation was performed with ten nurses and four physicians, respectively. The Unified Theory of Acceptance and Use of Technology (UTAUT) was used to analyze acceptance. RESULTS: We identified several potential benefits with the CDS prototype as well as some concerns that could be addressed by redesign. There was also concern about dependence on physician attitudes, as well as technical, organizational, and legal issues. CONCLUSIONS: Acceptance evaluation at a prototype stage provided rich data to improve the future design of a CDS prototype. Apart from design and development efforts, substantial organizational efforts are needed to enable the implementation and maintenance of a future CDS system.

Evaluation of Electronic Prescribing Decision Support System at a Tertiary Care Pediatric Hospital: The User Acceptance Perspective.

This study aims to evaluate pediatrician's acceptance, perception and use of Electronic Prescribing Decision Support Systems (EPDSS) at a tertiary care using Extended Technology Acceptance Model (TAM2). Qualitative research methodology was applied. Semi-structured questions were developed according to TAM2 model. Pediatricians perceived that the EPDSS is useful and they showed a favorable attitude. However, perceived ease of use and output quality appeared to affect use of EPDSS. Concerns were expressed about complicated screens, difficulty to read and view medication overview of the patient, the navigation requires many clicks and medication system don't meet their need. End users have difficulty of ordering drugs for ploy-clinical patients and they were unable to cancel or stop medications. Junior pediatricians were influenced by senior colleague since they can get better advice about medication order than the system. Applying TAM2 framework has revealed that pediatrician's attitude and acceptance of electronic prescribing system. This study has identified factors that are important for end user acceptance as well as suggestions for system improvement. Although pediatricians are positive to the usefulness of EPDSS, it appears there are some acceptance problems due to ease of use concern and usability issues of the system.

The effect of Computerized Physician Order Entry and decision support system on medication errors in the neonatal ward: experiences from an Iranian teaching hospital.

Medication dosing errors are frequent in neonatal wards. In an Iranian neonatal ward, a 7.5 months study was designed in three periods to compare the effect of Computerized Physician Order Entry (CPOE) without and with decision support functionalities in reducing non-intercepted medication dosing errors in antibiotics and anticonvulsants. Before intervention (Period 1), error rate was 53%, which did not significantly change after the implementation of CPOE without decision support (Period 2). However, errors were significantly reduced to 34% after that the decision support was added to the CPOE (Period 3; P < 0.001). Dose errors were more often intercepted than frequency errors. Over-dose was the most frequent type of medication errors and curtailed-interval was the least. Transcription errors did not reduce after the CPOE implementation. Physicians ignored alerts when they could not understand why they appeared. A suggestion is to add explanations about these reasons to increase physicians' compliance with the system's recommendations.

Physician order entry or nurse order entry? Comparison of two implementation strategies for a computerized order entry system aimed at reducing dosing medication errors.

BACKGROUND: Despite the significant effect of computerized physician order entry (CPOE) in reducing nonintercepted medication errors among neonatal inpatients, only a minority of hospitals have successfully implemented such systems. Physicians' resistance and users' frustration seem to be two of the most important barriers. One solution might be to involve nurses in the order entry process to reduce physicians' data entry workload and resistance. However, the effect of this collaborative order entry method in reducing medication errors should be compared with a strictly physician order entry method. OBJECTIVE: To investigate whether a collaborative order entry method consisting of nurse order entry (NOE) followed by physician verification and countersignature is as effective as a strictly physician order entry (POE) method in reducing nonintercepted dose and frequency medication errors in the neonatal ward of an Iranian teaching hospital. METHODS: A four-month prospective study was designed with two equal periods. During the first period POE was used and during the second period NOE was used. In both methods, a warning appeared when the dose or frequency of the prescribed medication was incorrect that suggested the appropriate dosage to the physicians. Physicians' responses to the warnings were recorded in a database and subsequently analyzed. Relevant paper-based and electronic medical records were reviewed to increase credibility. RESULTS: Medication prescribing for 158 neonates was studied. The rate of nonintercepted medication errors during the NOE period was 40% lower than during the POE period (rate ratio 0.60; 95% confidence interval [CI] .50, .71;P < .001). During the POE period, 80% of nonintercepted errors occurred at the prescription stage, while during the NOE period, 60% of nonintercepted errors occurred in that stage. Prescription errors decreased from 10.3% during the POE period to 4.6% during the NOE period (P < .001), and the number of warnings with which physicians complied increased from 44% to 68% respectively (P < .001). Meanwhile, transcription errors showed a nonsignificant increase from the POE period to the NOE period. The median error per patient was reduced from 2 during the POE period to 0 during the NOE period (P = .005). Underdose and curtailed and prolonged interval errors were significantly reduced from the POE period to the NOE period. The rate of nonintercepted overdose errors remained constant between the two periods. However, the severity of overdose errors was lower in the NOE period (P = .02). CONCLUSIONS: NOE can increase physicians' compliance with warnings and recommended dose and frequency and reduce nonintercepted medication dosing errors in the neonatal ward as effectively as POE or even better. In settings where there is major physician resistance to implementation of CPOE, and nurses are willing to participate in the order entry and are capable of doing so, NOE may be considered a beneficial alternative order entry method.

CPOE in Iran--a viable prospect? Physicians' opinions on using CPOE in an Iranian teaching hospital.

BACKGROUND: In recent years, the theory that on-line clinical decision support systems can improve patients' safety among hospitalised individuals has gained greater acceptance. However, the feasibility of implementing such a system in a middle or low-income country has rarely been studied. Understanding the current prescription process and a proper needs assessment of prescribers can act as the key to successful implementation. OBJECTIVES: The aim of this study was to explore physicians' opinions on the current prescription process, and the expected benefits and perceived obstacles to employ Computerised Physician Order Entry in an Iranian teaching hospital. METHODS: Initially, the interview guideline was developed through focus group discussions with eight experts. Then semi-structured interviews were held with 19 prescribers. After verbatim transcription, inductive thematic analysis was performed on empirical data. Forty hours of on-looker observations were performed in different wards to explore the current prescription process. RESULTS: The current prescription process was identified as a physician-centred, top-down, model, where prescribers were found to mostly rely on their memories as well as being overconfident. Some errors may occur during different paper-based registrations, transcriptions and transfers. Physician opinions on Computerised Physician Order Entry were categorised into expected benefits and perceived obstacles. Confidentiality issues, reduction of medication errors and educational benefits were identified as three themes in the expected benefits category. High cost, social and cultural barriers, data entry time and problems with technical support emerged as four themes in the perceived obstacles category. CONCLUSIONS: The current prescription process has a high possibility of medication errors. Although there are different barriers confronting the implementation and continuation of Computerised Physician Order Entry in Iranian hospitals, physicians have a willingness to use them if these systems provide significant benefits. A pilot study in a limited setting and a comprehensive analysis of health outcomes and economic indicators should be performed, to assess the merits of introducing Computerised Physician Order Entry with decision support capabilities in Iran.

Dynamic decision support graph--visualization of ANN-generated diagnostic indications of pathological conditions developing over time.

OBJECTIVES: A common objection to using artificial neural networks in clinical decision support systems is that the reasoning behind diagnostic indications cannot be sufficiently well explained. This paper presents a method for visualizing diagnostic indications generated from an artificial neural network-based decision support algorithm (ANN-algorithm) in conditions developing over time. METHODS: The main idea behind the method is first to calculate and graphically present the decision regions corresponding to the diagnostic indications given as output from the ANN-algorithm, in the space of two selected, clinically established 'display variables'. Secondly, the trajectory of time series measurement results of these, often biochemical markers, together with the respective 95% confidence intervals are superimposed on the decision regions. This will permit a nurse or clinician to grasp the diagnostic indication graphically at a glance. The indication is further presented in relation to clinical variables that the clinician is already familiar with, thus providing a sort of explanation. The predictive value of the indication is expressed by the proximity of the measurement result to the decision boundary, separating the decision regions, and by a numerically calculated individualized predictive value. RESULTS: The method is illustrated as applied to a previously published ANN-algorithm for the early ruling-in and ruling-out of acute myocardial infarction, using monitoring of measurement results of myoglobin and troponin-I in plasma. CONCLUSION: The method is appropriate when there is a limited number of clinically established variables, i.e. variables which the clinician is used to taking into account in clinical reasoning.

What they fill in today, may not be useful tomorrow: lessons learned from studying Medical Records at the Women hospital in Tabriz, Iran.

BACKGROUND: The medical record is used to document patient's medical history, illnesses and treatment procedures. The information inside is useful when all needed information is documented properly. Medical care providers in Iran have complained of low quality of Medical Records. This study was designed to evaluate the quality of the Medical Records at the university hospital in Tabriz, Iran. METHODS: In order to get a background of the quality of documentation, 300 Medical Records were randomly selected among all hospitalized patient during September 23, 2003 and September 22, 2004. Documentation of all records was evaluated using checklists. Then, in order to combine objective data with subjective, 10 physicians and 10 nurses who were involved in documentation of Medical Records were randomly selected and interviewed using two semi structured guidelines. RESULTS: Almost all 300 Medical Records had problems in terms of quality of documentation. There was no record in which all information was documented correctly and compatible with the official format in Medical Records provided by Ministry of Health and Medical Education. Interviewees believed that poor handwriting, missing of sheets and imperfect documentation are major problems of the Paper-based Medical Records, and the main reason was believed to be high workload of both physicians and nurses. CONCLUSION: The Medical Records are expected to be complete and accurate. Our study has unveiled that the Medical Records are not documented properly in the university hospital where the Medical Records are also used for educational purposes. Such incomplete Medical Records are not reliable resources for medical care too. Some influencing factors external to the structure of the Medical Records (i.e. human factors and work conditions) are involved.

Artificial neural network algorithms for early diagnosis of acute myocardial infarction and prediction of infarct size in chest pain patients.

BACKGROUND: To prospectively validate artificial neural network (ANN)-algorithms for early diagnosis of myocardial infarction (AMI) and prediction of 'major infarct' size in patients with chest pain and without ECG changes diagnostic for AMI. METHODS: Results of early and frequent Stratus CS measurements of troponin I (TnI) and myoglobin in 310 patients were used to validate four prespecified ANN-algorithms with use of cross-validation techniques. Two separate biochemical criteria for diagnosis of AMI were applied: TnI > or = 0.1 microg/L within 24 h ('TnI 0.1 AMI') and TnI > or = 0.4 microg/L within 24 h ('TnI 0.4 AMI'). To be considered clinically useful, the ANN-indications of AMI had to achieve a predefined positive predictive value (PPV) > or = 78% and a negative predictive value (NPV) > or = 94% at 2 h after admission. 'Major infarct' size was defined by peak levels of CK-MB within 24 h. RESULTS: For the best performing ANN-algorithms, the PPV and NPV for the indication of 'TnI 0.1 AMI' were 87% (p=0.009) and 99% (p=0.0001) at 2 h, respectively. For the indication of 'TnI 0.4 AMI', the PPV and NPV were 90% (p=0.006) and 99% (p=0.0004), respectively. Another ANN-algorithm predicted 'major AMI' at 2 h with a sensitivity of 96% and a specificity of 78%. Corresponding PPV and NPV were 73% and 97%, respectively. CONCLUSIONS: Specially designed ANN-algorithms allow diagnosis of AMI within 2 h of monitoring. These algorithms also allow early prediction of 'major AMI' size and could thus, be used as a valuable instrument for rapid assessment of chest pain patients.