Impact of diagnostic methods on efficacy estimation - a proof-of-principle based on historical examples.

INTRODUCTION: Accurate diagnostic methods are essential for evaluating treatment efficacy in clinical trials, including vaccine trials. Although a plethora of studies assessing novel or modified treatment options is available, clinical trials evaluating the sensitivity and specificity of diagnostic methods in compliance with the demands of drug registration trials are scarce. We assessed the accuracy of diagnostic methods in two vaccine trials conducted in 1995 and 2009 to demonstrate the impact of sensitivity and specificity on efficacy estimations. METHODS: We applied the sensitivity- and specificity-adjusted vaccine efficacy estimator of Lachenbruch for modelling the impact of test characteristics on the outcome of the two vaccine trials by varying diagnostic specificity. RESULTS: Because of non-ideal diagnostic sensitivity and specificity, the estimation of vaccine efficacy is compromised. We demonstrate the impact of diagnostic accuracy on efficacy estimations with increasing confidence limits. CONCLUSIONS: Because sensitivity and specificity less than one have a direct impact on efficacy estimations in clinical trials, evaluation of diagnostic methods should lead to a level of evidence comparable with the efficacy assessment of novel treatment options. Furthermore, statistical methods adjusted for sensitivity and specificity of diagnostic methods should be applied for efficacy estimations, or this lack of confidence has to be taken into account when interpreting the results of trials.

INTRODUCTION: Des méthodes de diagnostic précises sont essentielles pour évaluer l'efficacité du traitement dans les essais cliniques, y compris les essais de vaccins. Bien qu'une pléthore d'études évaluant des options de traitement nouvelles ou modifiées soit disponible, les essais cliniques évaluant la sensibilité et la spécificité des méthodes de diagnostic conformément aux exigences des essais pour l'enregistrement des médicaments sont rares. Nous avons évalué la précision des méthodes de diagnostic dans deux essais vaccinaux menés en 1995 et 2009 afin de démontrer l'impact de la sensibilité et de la spécificité sur les estimations d'efficacité. MÉTHODE: Nous avons appliqué l'estimateur d'efficacité vaccinale ajusté en fonction de la sensibilité et de la spécificité de Lachenbruch pour modéliser l'impact des caractéristiques des tests sur les résultats de deux essais vaccinaux en faisant varier la spécificité diagnostique. RÉSULTAT: En raison de la sensibilité et de la spécificité diagnostiques non idéales, l'estimation de l'efficacité du vaccin est compromise. Nous démontrons l'impact de la précision du diagnostic sur les estimations d'efficacité avec l'augmentation des limites de confiance. CONCLUSION: Etant donné que la sensibilité et la spécificité inférieures à un ont un impact direct sur les estimations d'efficacité dans les essais cliniques, l'évaluation des méthodes de diagnostic devrait conduire à un niveau d'évidence comparable à l'évaluation de l'efficacité de nouvelles options de traitement. En outre, des méthodes statistiques ajustées en fonction de la sensibilité et de la spécificité des méthodes de diagnostic doivent être appliquées pour les estimations de l'efficacité, ou alors ce manque de confiance devrait être pris en compte lors de l'interprétation des résultats des essais.

Evaluation of a clinical tool to test and adjust the programmed overnight basal profiles for insulin pump therapy: a pilot study.

OBJECTIVE: Clinical protocols for basal rate testing and adjustment are needed for effective insulin pump therapy. We evaluated the effects of a continuous glucose monitoring (CGM)-based semiautomated basal algorithm on glycemia. METHODS: We developed and piloted a basal rate analyzer that interpreted CGM data from overnight fasts and recommended dose changes for subsequent nights. Subjects uploaded data online using sensor-augmented pumps for evaluation by the analyzer after each of 5 overnight fasts conducted over 2 to 8 weeks. It was designed to be conservative and iterative, making changes that did not exceed 10% at each iteration. The standard deviation and interquartile range of CGM values from midnight to 7 am (SD12-7am and IQR12-7am) over 3 baseline and 3 postintervention nights, hypoglycemia incidence (CGM values <4.0 mmol/L), and glycated hemoglobin (A1C) were compared. RESULTS: Twenty subjects with mean ages of 38±13 years and A1C 7.6%±0.8% (60±8.7 mmol/mol) underwent the 5 iterations of basal assessments over 5±3 weeks. SD12-7am and IQR12-7am did not change from baseline to postintervention (1.57±0.8 to 1.63±0.8 mmol/L; p=0.35; 3.66±2.07 to 3.47±2.26 mmol/L; p=0.90). However, mean glucose values were lower between 2 to 3 am at baseline compared to postintervention; 3-night hypoglycemia incidence declined from 1.6±1.8 to 0.5±0.7 episodes (p=0.01), and A1C improved from 7.6%±0.8% to 7.4%±0.9% (60%±8.7% to 57%±9.8% mmol/mol; p=0.03). CONCLUSIONS: The use of a basal rate analyzer was associated with reduced hypoglycemia and improved A1C. However, overnight glycemic stability was not improved. Further research into the efficacy of the CGM-based semiautomated algorithm is warranted.