Does biomarker use in oncology improve clinical trial failure risk? A large-scale analysis.

PURPOSE: To date there has not been an extensive analysis of the outcomes of biomarker use in oncology. METHODS: Data were pooled across four indications in oncology drawing upon trial outcomes from www.clinicaltrials.gov: breast cancer, non-small cell lung cancer (NSCLC), melanoma and colorectal cancer from 1998 to 2017. We compared the likelihood drugs would progress through the stages of clinical trial testing to approval based on biomarker status. This was done with multi-state Markov models, tools that describe the stochastic process in which subjects move among a finite number of states. RESULTS: Over 10000 trials were screened, which yielded 745 drugs. The inclusion of biomarker status as a covariate significantly improved the fit of the Markov model in describing the drug trajectories through clinical trial testing stages. Hazard ratios based on the Markov models revealed the likelihood of drug approval with biomarkers having nearly a fivefold increase for all indications combined. A 12, 8 and 7-fold hazard ratio was observed for breast cancer, melanoma and NSCLC, respectively. Markov models with exploratory biomarkers outperformed Markov models with no biomarkers. CONCLUSION: This is the first systematic statistical evidence that biomarkers clearly increase clinical trial success rates in three different indications in oncology. Also, exploratory biomarkers, long before they are properly validated, appear to improve success rates in oncology. This supports early and aggressive adoption of biomarkers in oncology clinical trials.

Automated verbal autopsy classification: using one-against-all ensemble method and Naïve Bayes classifier.

Verbal autopsy (VA) deals with post-mortem surveys about deaths, mostly in low and middle income countries, where the majority of deaths occur at home rather than a hospital, for retrospective assignment of causes of death (COD) and subsequently evidence-based health system strengthening. Automated algorithms for VA COD assignment have been developed and their performance has been assessed against physician and clinical diagnoses. Since the performance of automated classification methods remains low, we aimed to enhance the Naïve Bayes Classifier (NBC) algorithm to produce better ranked COD classifications on 26,766 deaths from four globally diverse VA datasets compared to some of the leading VA classification methods, namely Tariff, InterVA-4, InSilicoVA and NBC. We used a different strategy, by training multiple NBC algorithms using the one-against-all approach (OAA-NBC). To compare performance, we computed the cumulative cause-specific mortality fraction (CSMF) accuracies for population-level agreement from rank one to five COD classifications. To assess individual-level COD assignments, cumulative partially-chance corrected concordance (PCCC) and sensitivity was measured for up to five ranked classifications. Overall results show that OAA-NBC consistently assigns CODs that are the most alike physician and clinical COD assignments compared to some of the leading algorithms based on the cumulative CSMF accuracy, PCCC and sensitivity scores. The results demonstrate that our approach improves the performance of classification (sensitivity) by between 6% and 8% compared with other VA algorithms. Population-level agreements for OAA-NBC and NBC were found to be similar or higher than the other algorithms used in the experiments. Although OAA-NBC still requires improvement for individual-level COD assignment, the one-against-all approach improved its ability to assign CODs that more closely resemble physician or clinical COD classifications compared to some of the other leading VA classifiers.

Practice does not always make perfect: need for selection curricula in modern surgical training.

BACKGROUND: It is hypothesized that not all surgical trainees are able to reach technical competence despite ongoing practice. The objectives of the study were to assess a trainees' ability to reach technical competence by assessing learning patterns of the acquisition of surgical skills. Furthermore, it aims to determine whether individuals' learning patterns were consistent across a range of open and laparoscopic tasks of variable difficulty. METHODS: Sixty-five preclinical medical students participated in a training curriculum with standardized feedback over forty repetitions of the following laparoscopic and open technical tasks: peg transfer (PT), circle cutting (CC), intracorporeal knot tie (IKT), one-handed tie, and simulated laparotomy closure. Data mining techniques were used to analyze the prospectively collected data and stratify the students into four learning clusters. Performance was compared between groups, and learning curve characteristics unique to trainees who have difficulty reaching technical competence were quantified. RESULTS: Top performers (22-35%) and high performers (32-42%) reached proficiency in all tasks. Moderate performers (25-37%) reached proficiency for all open tasks but not all laparoscopic tasks. Low performers (8-15%) failed to reach proficiency in four of five tasks including all laparoscopic tasks (PT 7.8%; CC 9.4%; IKT 15.6%). Participants in lower performance clusters demonstrated sustained performance disadvantage across tasks, with widely variable learning curves and no evidence of progression towards a plateau phase. CONCLUSIONS: Most students reached proficiency across a range of surgical tasks, but low-performing trainees failed to reach competence in laparoscopic tasks. With increasing use of laparoscopy in surgical practice, screening potential candidates to identify the lowest performers may be beneficial.

Empirical Models of Social Learning in a Large, Evolving Network.

This paper advances theories of social learning through an empirical examination of how social networks change over time. Social networks are important for learning because they constrain individuals' access to information about the behaviors and cognitions of other people. Using data on a large social network of mobile device users over a one-month time period, we test three hypotheses: 1) attraction homophily causes individuals to form ties on the basis of attribute similarity, 2) aversion homophily causes individuals to delete existing ties on the basis of attribute dissimilarity, and 3) social influence causes individuals to adopt the attributes of others they share direct ties with. Statistical models offer varied degrees of support for all three hypotheses and show that these mechanisms are more complex than assumed in prior work. Although homophily is normally thought of as a process of attraction, people also avoid relationships with others who are different. These mechanisms have distinct effects on network structure. While social influence does help explain behavior, people tend to follow global trends more than they follow their friends.

Predictive Modeling of Implantation Outcome in an In Vitro Fertilization Setting: An Application of Machine Learning Methods.

BACKGROUND: Multiple embryo transfers in in vitro fertilization (IVF) treatment increase the number of successful pregnancies while elevating the risk of multiple gestations. IVF-associated multiple pregnancies exhibit significant financial, social, and medical implications. Clinicians need to decide the number of embryos to be transferred considering the tradeoff between successful outcomes and multiple pregnancies. OBJECTIVE: To predict implantation outcome of individual embryos in an IVF cycle with the aim of providing decision support on the number of embryos transferred. DESIGN: Retrospective cohort study. DATA SOURCE: Electronic health records of one of the largest IVF clinics in Turkey. The study data set included 2453 embryos transferred at day 2 or day 3 after intracytoplasmic sperm injection (ICSI). Each embryo was represented with 18 clinical features and a class label, +1 or -1, indicating positive and negative implantation outcomes, respectively. METHODS: For each classifier tested, a model was developed using two-thirds of the data set, and prediction performance was evaluated on the remaining one-third of the samples using receiver operating characteristic (ROC) analysis. The training-testing procedure was repeated 10 times on randomly split (two-thirds to one-third) data. The relative predictive values of clinical input characteristics were assessed using information gain feature weighting and forward feature selection methods. RESULTS: The naïve Bayes model provided 80.4% accuracy, 63.7% sensitivity, and 17.6% false alarm rate in embryo-based implantation prediction. Multiple embryo implantations were predicted at a 63.8% sensitivity level. Predictions using the proposed model resulted in higher accuracy compared with expert judgment alone (on average, 75.7% and 60.1%, respectively). CONCLUSIONS: A machine learning-based decision support system would be useful in improving the success rates of IVF treatment.

Physician experience in performing embryo transfers may affect outcome.

The distribution of six physicians' pregnancy rates with cycle and patient demographics was investigated for 2,212 transfer cycles. The results indicate that when the patient and cycle characteristics are compromised, the level of physician experience may determine the outcome of embryo transfers.

A frequency based encoding technique for transformation of categorical variables in mixed IVF dataset.

Implantation prediction of in-vitro fertilization (IVF) embryos is critical for the success of the treatment. In this study, Support Vector Machine (SVM) method has been used on an original IVF dataset for classification of embryos according to implantation potentials. The dataset we analyzed includes both categorical and continuous feature values. Transformation of categorical variables into numeric attributes is an important pre-processing stage for SVM affecting the performance of the classification. We have proposed a frequency based encoding technique for transformation of categorical variables. Experimental results revealed that, the proposed technique significantly improved the performance of IVF implantation prediction in terms of Area Under ROC curve (0.712+/-0.032) compared to common binary encoding and expert judgement based transformation methods (0.676+/-0.033 and 0.696 +/- 0.024, respectively).