Validation and Reproducibility of the Updated French Causality Assessment Method: an Evaluation by Pharmacovigilance Centres & Pharmaceutical Companies.

OBJECTIVE: Assess the validity and reproducibility of the updated version of the French causality assessment method in conditions approaching real-life use. METHODS: A random sample of 31 drug-event pairs from the French pharmacovigilance database was assessed by the consensual judgement of three experts (gold standard). Separately, a team from a pharmacovigilance centre (PhVC) and another from a pharmaceutical company assessed these pairs using the current method, then with the updated method. To test the inter- and intra-rater reproducibility, two seniors and two juniors from a PhVC and a pharmaceutical company assessed the pairs twice with the updated method. A weighted kappa coefficient was used to measure the agreement of the two causality assessment methods with the consensual expert judgement (validity) as well as the agreement of the updated causality assessment over time (intra-rater reproducibility) and between evaluators (inter-rater reproducibility). RESULTS: Agreement between the current method and consensual expert judgement was fair for the PhVC team (weighted kappa [Kw] 0.33) and moderate for the pharmaceutical company team (Kw 0.41). For the updated method, agreement was better for both the PhVC (Kw 0.58) and the pharmaceutical company (Kw 0.52) teams. The inter- and intra-rater reproducibility of the updated method based on the intrinsic imputability was satisfactory overall (Kw 0.30-0.91). Discrepancies between evaluations from PhVC and pharmaceutical companies were observed with the updated method. CONCLUSION: The updated method performed better than the current one for drug causality assessment, suggesting that it should be used in routine pharmacovigilance.

Updating the French method for the causality assessment of adverse drug reactions.

The Imputability Working Group (CRI) updated the French drug reaction causality assessment method. This tripartite group is made up of staff from the French network of regional pharmacovigilance centres, pharmaceutical companies, and the French National Agency for the Safety of Medicines and Health Products (ANSM). After reviewing the strengths and weaknesses of the previous method, several ideas for improvement were proposed: a better-worded and more discriminating scale for certain chronological and semiological criteria, a larger scale for the intrinsic score (increased from 5 to 7 levels), a new bibliographical scale to differentiate between expected and unexpected adverse drug reactions, and a new informativeness scale.

[Not Available]. / Réactualisation de la méthode française d'imputabilité des effets indésirables des médicaments.

A tripartite group, entitled « CRI ¼ (for Cercle de réflexion sur l'imputabilité), involving French pharmacovigilance staff from the French network of the Regional centers of pharmacovigilance, from pharmaceutical companies, and from French Health Authorities (Agence française de sécurité sanitaire des produits de santé) has worked to update the French drug reaction assessment method. Following assessment of strengths and weaknesses of the previous method, several points for improvement are proposed : a more precise wording and a more discriminating scale for some of the chronological and semiological criteria, a wider distribution of the intrinsic score from 5 to 7 levels, a new bibliographic scale for differentiating expected and unexpected adverse drug reactions, and a new informativness scale. This updated method would lead to a more relevant assessment of relationship between a drug and the occurrence of an adverse reaction. Furthermore, this method is a teaching tool to assess the level of relationship between a drug and the occurrence of an adverse reaction.

[Update of the French drug reaction assessment method]. / Réactualisation de la méthode française d'imputabilité des effets indésirables des médicaments.

A tripartite group, entitled « CRI ¼ (for Cercle de réflexion sur l'imputabilité), involving French pharmacovigilance staff from the French network of the Regional centers of pharmacovigilance, from pharmaceutical companies, and from French Health Authorities (Agence française de sécurité sanitaire des produits de santé) has worked to update the French drug reaction assessment method. Following assessment of strengths and weaknesses of the previous method, several points for improvement are proposed: a more precise wording and a more discriminating scale for some of the chronological and semiological criteria, a wider distribution of the intrinsic score from 5 to 7 levels, a new bibliographic scale for differentiating expected and unexpected adverse drug reactions, and a new informativness scale. This updated method would lead to a more relevant assessment of relationship between a drug and the occurrence of an adverse reaction. Furthermore, this method is a teaching tool to assess the level of relationship between a drug and the occurrence of an adverse reaction.

A new method for assessing drug causation provided agreement with experts' judgment.

BACKGROUND AND OBJECTIVE: The many methods proposed for causality assessment of adverse drug reaction (ADR) generally rely on algorithms. They have no clear relationship to probabilities, however, a situation we attempted to improve. STUDY DESIGN AND SETTING: Thirty ADR cases corresponding to 32 suspect drugs were randomly selected from the French pharmacovigilance database. The statistical weighting was performed by using a multilinear regression with logit(p) as the dependent variable and seven judgment criteria as independent variables. The best model (i.e., giving the best correlation with the gold standard) was retained for the new causality assessment method. RESULTS: The weights [logit(p)] for the 21 choices, on average three for each of the seven criteria, ranged from -3.95 to 0.86, secondarily rounded to multiples of 0.5. The correlation between the probability obtained from the final method and the gold standard was quite good (R(2) = .92). CONCLUSION: This method based on the rational weighting of seven causality criteria is straightforward to use and provides very good agreement with experts' judgment. Moreover, unlike most classical algorithms, it respects one basic rule of probabilities-namely, a symmetrical probability distribution for drug causation around the .5 neutral position (maximum uncertainty).

The bilateral superficial cervical plexus block with 0.75% ropivacaine administered before or after surgery does not prevent postoperative pain after total thyroidectomy.

BACKGROUND AND OBJECTIVES: Patients undergoing thyroid surgery need postoperative pain management. Bilateral superficial cervical plexus block by administration of 0.25% bupivacaine with 1:200000 epinephrine at the end of surgery has been shown to improve postoperative analgesia. The objective of this study was to assess the analgesic efficacy in the first 36 postoperative hours after total thyroidectomy of bilateral superficial cervical plexus block with 0.75% ropivacaine administered before the incision or on completion of the surgical procedure. METHODS: We performed a prospective double-blinded, randomized controlled trial that compared 3 parallel groups: the CONT group did not receive any block, the PRE group received bilateral superficial cervical plexus block before surgery while under general anesthesia, and the POST group received bilateral superficial cervical plexus block after surgery while under general anesthesia. The study included 111 patients (37 in each group). Postoperative pain was assessed every 4 hours by use of a 0 to 10 numeric rating scale. All patients received paracetamol every 6 hours. Morphine was administered following a standardized protocol if the numeric rating scale was 4 or higher. The main outcome variables were the proportion of patients given morphine during the 36 hours period, pain intensity scores, and morphine consumption. RESULTS: No intergroup differences were observed in terms of percentage of patients who required morphine, morphine delivery, pain scores, and intraoperative opioid consumption. CONCLUSIONS: Bilateral superficial cervical plexus block with 0.75% ropivacaine administered before or after surgery does not improve postoperative analgesia after total thyroidectomy.

Comparison of three methods (an updated logistic probabilistic method, the Naranjo and Liverpool algorithms) for the evaluation of routine pharmacovigilance case reports using consensual expert judgement as reference.

BACKGROUND: An updated probabilistic causality assessment method and the Liverpool algorithm presented as an improved version of the Naranjo algorithm, one of the most used and accepted causality assessment methods, have recently been proposed. OBJECTIVE: In order to test the validity of the probabilistic method in routine pharmacovigilance, results provided by the Naranjo and Liverpool algorithms, as well as the updated probabilistic method, were each compared with a consensual expert judgement taken as reference. METHODS: A sample of 59 drug-event pairs randomly sampled from spontaneous reports to the French pharmacovigilance system was assessed by expert judgement until reaching consensus and by members of a pharmacovigilance unit using the updated probabilistic method, the Naranjo and Liverpool algorithms. Probabilities given by the probabilistic method, and categories obtained by both the Naranjo and the Liverpool algorithms were compared as well as their sensitivity, specificity, positive and negative predictive values. RESULTS: The median probability for drug causation given by the consensual expert judgement was 0.70 (inter-quartile range, IQR 0.54-0.84) versus 0.77 (IQR 0.54-0.91) for the probabilistic method. For the Naranjo algorithm, the 'possible' causality category was predominant (61 %), followed by 'probable' (35 %), 'doubtful', and 'almost certain' categories (2 % each). Category distribution obtained with the Liverpool algorithm was similar to that obtained by the Naranjo algorithm with a majority of 'possible' (61 %) and 'probable' (30 %) followed by 'definite' (7 %) and 'unlikely' (2 %). For the probabilistic method, sensitivity, specificity, positive and negative predictive values were 0.96, 0.56, 0.92 and 0.71, respectively. For the Naranjo algorithm, depending on whether the 'possible' category was considered in favour or in disfavour of drug causation, sensitivity was, respectively, 1 or 0.42, specificity 0.11 or 0.89, negative predictive value 1 or 0.22 and positive predictive value 0.86 or 0.95; results were identical for the Liverpool algorithm. CONCLUSION: The logistic probabilistic method gave results closer to the consensual expert judgment than either the Naranjo or Liverpool algorithms whose performance were strongly dependent on the meaning given to the 'possible' category. Owing to its good sensitivity and positive predictive value and by providing results as continuous probabilities, the probabilistic method seems worthy to use for a trustable assessment of adverse drug reactions in routine practice.

An updated method improved the assessment of adverse drug reaction in routine pharmacovigilance.

OBJECTIVE: Updating a logistic causality assessment method to improve its agreement with consensual expert judgment (CEJ). STUDY DESIGN AND SETTING: A random sample of 53 drug-event pairs from a pharmacovigilance database were evaluated independently by CEJ and by a group of experts in pharmacovigilance using the logistic method. Causes of disagreement between both approaches were analyzed, and changes in the assessment of some criteria of the logistic method were proposed and tested in models. The model giving results closest to the CEJ was retained and compared with the initial version on another set of drug-event pairs. RESULTS: Finally, only the criterion "Search for nondrug cause" was changed into "Search for other causes." The assessment not investigated, possible other cause decreased the probability of drug causation instead of being neutral, whereas the assessment not applicable, not required remained neutral. This new version presents much improved specificity (0.56 vs. 0.33), relatively good sensitivity (0.96), and positive and negative predictive values (0.92 and 0.71). CONCLUSION: The updated logistic method presented here improves the initial version that had poor specificity and tended to overestimate drug causation. This new version presents satisfactory characteristics to be used in routine pharmacovigilance.

Comparison of three methods (consensual expert judgement, algorithmic and probabilistic approaches) of causality assessment of adverse drug reactions: an assessment using reports made to a French pharmacovigilance centre.

BACKGROUND: Different methods have been proposed for assessing a possible causal link between a drug treatment and an adverse event in individual patients. They approximately belong to three main categories: expert judgement, operational algorithms and probabilistic approaches. OBJECTIVE: To compare, in a set of actual drug adverse event reports, three different methods for assessing drug causality, each belonging to one of the three main categories: expert judgement, the algorithm used by the French pharmacovigilance centres since 1985, and a novel method based on the logistic function. METHODS: Fifty drug-event pairs were randomly sampled from the database of the Bordeaux pharmacovigilance centre, France. To serve as the gold standard, the probability for drug causation, from 0 to 1, was first determined for each drug-event pair by a panel of senior experts until consensus was reached. Causality was then assessed by members of the Bordeaux pharmacovigilance centre by using the French algorithm and the logistic method. Results expressed as a probability with the logistic method and as a score from 0 to 4 with the French algorithm were then compared with consensual expert judgement, as were the sensitivity, specificity and positive and negative predictive values. RESULTS: Probabilities ranged from 0.08 to 0.99 (median 0.58; mean 0.60) for experts versus 0.18-0.88 (median 0.73; mean 0.67) for the logistic method. Consensual expert judgement was not discriminant (p = 0.50) in ten cases. For the algorithm, only three of five causality scores were found, doubtful scores being clearly predominant (74%) followed by possible (16%) and probable (10%) scores. Sensitivity and specificity were 0.96 and 0.42, respectively, for the logistic method versus 0.42 and 0.92 for the algorithm. Positive and negative predictive values were 0.78 and 0.83, respectively, for the logistic method versus 0.92 and 0.42 for the algorithm. CONCLUSIONS: Agreement between the three approaches was poor, and only satisfactory for drug events judged as drug-induced by consensual expert judgement. The logistic method showed high sensitivity at the expense of poor specificity. Conversely, the algorithm had poor sensitivity but good specificity. The comparatively good sensitivity and positive predictive values of the logistic method suggest that it may be more useful in the routine or automated assessment of case reports of suspected but still unknown adverse drug reactions. With a substantial rate of false positives relative to true negatives (low specificity), the logistic method does not replace, but can be complemented by, critical clinical assessment of individual cases in evaluating drug-related risk.

Usefulness of adrenal scintigraphy in the follow-up of adrenocortical incidentalomas: a prospective multicenter study.

OBJECTIVES: Prognostic factors for progression of benign adrenocortical adenomas (AI) remain poorly known. We assessed the usefulness of (131)I-6-beta-iodomethylnorcholesterol scintigraphy (IMS) to predict the occurrence of adrenal hyperfunction or mass enlargement. DESIGN: Fifty-one consecutive inpatients with unilateral AI and normal 24-h urinary free cortisol (UFC) were enrolled in a multicenter observational prospective study to investigate the relationship between the scintigraphic pattern and the progression of biological abnormalities of the hypothalamo-pituitary-adrenal axis or tumor size. RESULTS: Biochemically defined 'subclinical' Cushing's syndrome (SCS) was found at baseline in 47% of patients. Unilateral uptake (UU) was significantly associated with SCS (P<0.05). During the follow-up (4.3+/-1.6-year): 53% of patients showed unchanged hormonal evaluation, 29% displayed intermittent SCS and 18% showed definitive hormonal progression of SCS but without overt biochemical hypercortisolism. UU was associated with persistence of SCS and hormonal progression (P<0.01). In multivariate analysis, UU and impaired 1 mg dexamethasone suppression were independently associated with hormonal progression. Three patients with UU developed clinical CS despite persistently normal UFC. Tumor size increased in 10% patients and was not associated with any scintigraphic pattern. CONCLUSION: Evolution of SCS toward overt biochemical CS in patients with AI is a rare event during a 4-year follow-up. UU is predictive for the occurrence of SCS, its persistence and progression within the spectrum of SCS. Further studies aiming to establish the clinical consequences of SCS are needed to recommend IMS as a complementary evaluation in patients with AI and biochemical SCS.

Inter-expert agreement of seven criteria in causality assessment of adverse drug reactions.

AIMS: To evaluate agreement between five senior experts when assessing seven causality criteria and the probability of drug causation. METHODS: A sample of 31 adverse event-drug pairs was constituted. For each pair, five experts separately assessed (i) the probability of drug causation, which was secondarily divided into seven causality levels: ruled out (0-0.05), unlikely (0.06-0.25), doubtful (0.26-0.45), indeterminate (0.46-0.55), plausible (0.56-0.75), likely (0.76-0.95), and certain (0.96-1); and (ii) seven causality criteria. To test discrepancies between experts, the kappa index was used. RESULTS: The agreement of the five experts was very poor (kappa = 0.05) for the probability of drug causation. Among the seven levels of causality, only 'doubtful' showed a significant rate of agreement (kappa = 0.32, P < 0.001). For all criteria, the kappa index was significant except for the item 'risk(s) factor(s)' (kappa = 0.09). Agreement between experts was good (0.64, P < 0.001) only for the criterion 'reaction at site of application or toxic plasma concentration of the drug or validated test'. However, the rate of agreement with kappa indices of the causality criteria ranged from 0.12 to 0.38. CONCLUSIONS: This study confirms that in the absence of an operational procedure, agreement between experts is low. This should be considered when designing a causality assessment method. In particular, criteria inducing a low level of agreement should have their weight reduced.

Agreement of expert judgment in causality assessment of adverse drug reactions.

BACKGROUND: Global introspection is, with operational algorithms and Bayes' theorem, one of the three main approaches used to assess the causal relationship between a drug treatment and the occurrence of an adverse event. OBJECTIVE: To analyze and compare the judgments of five senior experts using global introspection about drug causation on a random set of putative adverse drug reactions. METHODS: A random sample of 150 drug-effect pairs was constituted. For each pair, five senior experts had to independently assess the probability of drug causation from 0 to 1 by using a 100 mm visual analog scale (VAS). For analysis, those probabilities were secondarily split into seven levels of causality: excluded (0-0.05); unlikely (0.06-0.25); doubtful (0.26-0.45); unassessable/unclassifiable (0.46-0.55); plausible (0.56-0.75); likely (0.75-0.95); and certain (0.95-1). Agreement among the five experts was assessed using kappa coefficients (kappa). RESULTS: The overall agreement between experts was poor (kappa=0.20), although significantly different from chance, and varied according to the level of causality. It was lower for the unlikely, doubtful, unassessable/unclassifiable, and plausible categories (kappa=0.03, 0.03, -0.01, and 0.13, respectively) than for VAS extremes: excluded, likely, and certain (kappa=0.40, 0.32, and 0.30, respectively). CONCLUSION: This study confirms that experts express marked disagreements when assessing drug causality independently. The agreement rate was lower for intermediate levels of causality, especially when strong evidence was lacking for confirming or ruling out drug causality. Therefore, in a decision-making context, a step-by-step consensual approach such as the Delphi method seems necessary to make the assessment of such cases more reliable.