What they forgot to tell you about machine learning with an application to pharmaceutical manufacturing.

Predictive models (a.k.a. machine learning models) are ubiquitous in all stages of drug research, safety, development, manufacturing, and marketing. The results of these models are used inside and outside of pharmaceutical companies for the purpose of understanding scientific processes and for predicting characteristics of new samples or patients. While there are many resources that describe such models, there are few that explain how to develop a robust model that extracts the highest possible performance from the available data, especially in support of pharmaceutical applications. This tutorial will describe pitfalls and best practices for developing and validating predictive models with a specific application to a monitoring a pharmaceutical manufacturing process. The pitfalls and best practices will be highlighted to call attention to specific points that are not generally discussed in other resources.

Analysis of chemometric models applied to Raman spectroscopy for monitoring key metabolites of cell culture.

The Food and Drug Administration (FDA) initiative of Process Analytical Technology (PAT) encourages the monitoring of biopharmaceutical manufacturing processes by innovative solutions. Raman spectroscopy and the chemometric modeling tool partial least squares (PLS) have been applied to this aim for monitoring cell culture process variables. This study compares the chemometric modeling methods of Support Vector Machine radial (SVMr), Random Forests (RF), and Cubist to the commonly used linear PLS model for predicting cell culture components-glucose, lactate, and ammonia. This research is performed to assess whether the use of PLS as standard practice is justified for chemometric modeling of Raman spectroscopy and cell culture data. Model development data from five small-scale bioreactors (2 × 1 L and 3 × 5 L) using two Chinese hamster ovary (CHO) cell lines were used to predict against a manufacturing scale bioreactor (2,000 L). Analysis demonstrated that Cubist predictive models were better for average performance over PLS, SVMr, and RF for glucose, lactate, and ammonia. The root mean square error of prediction (RMSEP) of Cubist modeling was acceptable for the process concentration ranges of glucose (1.437 mM), lactate (2.0 mM), and ammonia (0.819 mM). Interpretation of variable importance (VI) results theorizes the potential advantages of Cubist modeling in avoiding interference of Raman spectral peaks. Predictors/Raman wavenumbers (cm-1 ) of interest for individual variables are X1139-X1141 for glucose, X846-X849 for lactate, and X2941-X2943 for ammonia. These results demonstrate that other beneficial chemometric models are available for use in monitoring cell culture with Raman spectroscopy.

Automated plaque analysis for the prognostication of major adverse cardiac events.

OBJECTIVE: The purpose of this study is to assess the value of an automated model-based plaque characterization tool for the prediction of major adverse cardiac events (MACE). METHODS: We retrospectively included 45 patients with suspected coronary artery disease of which 16 (33%) experienced MACE within 12 months. Commercially available plaque quantification software was used to automatically extract quantitative plaque morphology: lumen area, wall area, stenosis percentage, wall thickness, plaque burden, remodeling ratio, calcified area, lipid rich necrotic core (LRNC) area and matrix area. The measurements were performed at all cross sections, spaced at 0.5 mm, based on fully 3D segmentations of lumen, wall, and each tissue type. Discriminatory power of these markers and traditional risk factors for predicting MACE were assessed. RESULTS: Regression analysis using clinical risk factors only resulted in a prognostic accuracy of 63% with a corresponding area under the curve (AUC) of 0.587. Based on our plaque morphology analysis, minimal cap thickness, lesion length, LRNC volume, maximal wall area/thickness, the remodeling ratio, and the calcium volume were included into our prognostic model as parameters. The use of morphologic features alone resulted in an increased accuracy of 77% with an AUC of 0.94. Combining both clinical risk factors and morphological features in a multivariate logistic regression analysis increased the accuracy to 87% with a similar AUC of 0.924. CONCLUSION: An automated model based algorithm to evaluate CCTA-derived plaque features and quantify morphological features of atherosclerotic plaque increases the ability for MACE prognostication significantly compared to the use of clinical risk factors alone.

Validation of Sysmex XT-2000iV analyzer-generated quantitative bone marrow differential counts in cynomolgus monkeys, Beagle dogs, and CD-1 mice.

BACKGROUND: In a previous study, the validation of rat bone marrow (BM) collection, processing, and analysis using the Sysmex XT-2000iV (Sysmex Corporation, Kobe, Japan) hematology analyzer showed that the Sysmex hematology analyzer produced BM differential counts that were comparable to those obtained with microscopic differential counts. OBJECTIVE: This study was conducted to expand the validation of the Sysmex TNCC (total nucleated cell count) and 5-part BM differential in cynomolgus monkeys, Beagle dogs, and CD-1 mice, which are alternate species that are also frequently used in preclinical safety studies. METHODS: The Sysmex 5-part BM differential counts were generated with a two-step process, whereby proliferating and maturing erythroid and myeloid cells were determined by preset gating and lymphocytes were determined using species-specific B- and T-lymphocyte antibodies and a magnetic cell-sorting method (MACS). Agreement with microscopic myelograms with 500-cell differential counts was determined from BM suspensions of 62 cynomolgus monkeys, 47 Beagle dogs, and 44 CD-1 mice. RESULTS: The correlation coefficients between methods for myeloid to erythroid (M:E) ratios in all three species was > 0.928. The Bland-Altman differences between methods were approximately ± 0.3 units for the M:E ratio in dogs and mice, and +0.6 and -0.4 in monkeys. The upper limits of agreement for all three species were ≤7% for maturing myeloid cells, ≤6% for maturing erythroid cells, and ≤4% for proliferating myeloid cells, proliferating erythroid cells, and lymphocytes. CONCLUSIONS: The Sysmex XT-2000iV produces an automated M:E ratio and a 5-part differential count equivalent to microscopic differential counts in cynomolgus monkeys, Beagle dogs, and CD-1 mice.

Multivariate analysis of cytokine profiles in pregnancy complications.

PROBLEM: The immunoregulation to tolerate the semiallogeneic fetus during pregnancy includes a harmonious dynamic balance between anti- and pro-inflammatory cytokines. Several earlier studies reported significantly different levels and/or ratios of several cytokines in complicated pregnancy as compared to normal pregnancy. However, as cytokines operate in networks with potentially complex interactions, it is also interesting to compare groups with multi-cytokine data sets, with multivariate analysis. Such analysis will further examine how great the differences are, and which cytokines are more different than others. METHODS: Various multivariate statistical tools, such as Cramer test, classification and regression trees, partial least squares regression figures, 2-dimensional Kolmogorov-Smirmov test, principal component analysis and gap statistic, were used to compare cytokine data of normal vs anomalous groups of different pregnancy complications. RESULTS: Multivariate analysis assisted in examining if the groups were different, how strongly they differed, in what ways they differed and further reported evidence for subgroups in 1 group (pregnancy-induced hypertension), possibly indicating multiple causes for the complication. CONCLUSION: This work contributes to a better understanding of cytokines interaction and may have important implications on targeting cytokine balance modulation or design of future medications or interventions that best direct management or prevention from an immunological approach.

Identification of novel biomarkers to monitor ß-cell function and enable early detection of type 2 diabetes risk.

A decline in ß-cell function is a prerequisite for the development of type 2 diabetes, yet the level of ß-cell function in individuals at risk of the condition is rarely measured. This is due, in part, to the fact that current methods for assessing ß-cell function are inaccurate, prone to error, labor-intensive, or affected by glucose-lowering therapy. The aim of the current study was to identify novel circulating biomarkers to monitor ß-cell function and to identify individuals at high risk of developing ß-cell dysfunction. In a nested case-control study from the Relationship between Insulin Sensitivity and Cardiovascular disease (RISC) cohort (n = 1157), proteomics and miRNA profiling were performed on fasting plasma samples from 43 individuals who progressed to impaired glucose tolerance (IGT) and 43 controls who maintained normal glucose tolerance (NGT) over three years. Groups were matched at baseline for age, gender, body mass index (BMI), insulin sensitivity (euglycemic clamp) and ß-cell glucose sensitivity (mathematical modeling). Proteomic profiling was performed using the SomaLogic platform (Colorado, USA); miRNA expression was performed using a modified RT-PCR protocol (Regulus Therapeutics, California, USA). Results showed differentially expressed proteins and miRNAs including some with known links to type 2 diabetes, such as adiponectin, but also novel biomarkers and pathways. In cross sectional analysis at year 3, the top differentially expressed biomarkers in people with IGT/ reduced ß-cell glucose sensitivity were adiponectin, alpha1-antitrypsin (known to regulate adiponectin levels), endocan, miR-181a, miR-342, and miR-323. At baseline, adiponectin, cathepsin D and NCAM.L1 (proteins expressed by pancreatic ß-cells) were significantly lower in those that progressed to IGT. Many of the novel prognostic biomarker candidates were within the epithelial-mesenchymal transition (EMT) pathway: for example, Noggin, DLL4 and miR-181a. Further validation studies are required in additional clinical cohorts and in patients with type 2 diabetes, but these results identify novel pathways and biomarkers that may have utility in monitoring ß-cell function and/ or predicting future decline, allowing more targeted efforts to prevent and intercept type 2 diabetes.

Novel Methods for Prioritizing "Close-In" Analogs from Structure-Activity Relationship Matrices.

Here we describe the development of novel methods for compound evaluation and prioritization based on the structure-activity relationship matrix (SARM) framework. The SARM data structure allows automatic and exhaustive extraction of SAR patterns from data sets and their organization into a chemically intuitive scaffold/functional-group format. While SARMs have been used in the retrospective analysis of SAR discontinuity and identifying underexplored regions of chemistry space, there have been only a few attempts to apply SARMs prospectively in the prioritization of "close-in" analogs. In this work, three new ways of prioritizing virtual compounds based on SARMs are described: (1) matrix pattern-based prioritization, (2) similarity weighted, matrix pattern-based prioritization, and (3) analysis of variance based prioritization (ANV). All of these methods yielded high predictive power for six benchmark data sets (prediction accuracy R2 range from 0.63 to 0.82), yielding confidence in their application to new design ideas. In particular, the ANV method outperformed the previously reported SARM based method for five out of the six data sets tested. The impact of various SARM parameters were investigated and the reasons why SARM-based compound prioritization methods provide higher predictive power are discussed.

Predictive Modeling of Response to Pregabalin for the Treatment of Neuropathic Pain Using 6-Week Observational Data: A Spectrum of Modern Analytics Applications.

PURPOSE: This post hoc analysis used 11 predictive models of data from a large observational study in Germany to evaluate potential predictors of achieving at least 50% pain reduction by week 6 after treatment initiation (50% pain response) with pregabalin (150-600 mg/d) in patients with neuropathic pain (NeP). METHODS: The potential predictors evaluated included baseline demographic and clinical characteristics, such as patient-reported pain severity (0 [no pain] to 10 [worst possible pain]) and pain-related sleep disturbance scores (0 [sleep not impaired] to 10 [severely impaired sleep]) that were collected during clinic visits (baseline and weeks 1, 3, and 6). Baseline characteristics were also evaluated combined with pain change at week 1 or weeks 1 and 3 as potential predictors of end-of-treatment 50% pain response. The 11 predictive models were linear, nonlinear, and tree based, and all predictors in the training dataset were ranked according to their variable importance and normalized to 100%. FINDINGS: The training dataset comprised 9187 patients, and the testing dataset had 6114 patients. To adjust for the high imbalance in the responder distribution (75% of patients were 50% responders), which can skew the parameter tuning process, the training set was balanced into sets of 1000 responders and 1000 nonresponders. The predictive modeling approaches that were used produced consistent results. Baseline characteristics alone had fair predictive value (accuracy range, 0.61-0.72; κ range, 0.17-0.30). Baseline predictors combined with pain change at week 1 had moderate predictive value (accuracy, 0.73-0.81; κ range, 0.37-0.49). Baseline predictors with pain change at weeks 1 and 3 had substantial predictive value (accuracy, 0.83-0.89; κ range, 0.54-0.71). When variable importance across the models was estimated, the best predictor of 50% responder status was pain change at week 3 (average importance 100.0%), followed by pain change at week 1 (48.1%), baseline pain score (14.1%), baseline depression (13.9%), and using pregabalin as a monotherapy (11.7%). IMPLICATIONS: The finding that pain changes by week 1 or weeks 1 and 3 are the best predictors of pregabalin response at 6 weeks suggests that adhering to a pregabalin medication regimen is important for an optimal end-of-treatment outcome. Regarding baseline predictors alone, considerable published evidence supports the importance of high baseline pain score and presence of depression as factors that can affect treatment response. Future research would be required to elucidate why using pregabalin as a monotherapy also had more than a 10% variable importance as a potential predictor.

Lost in translation: The search for an in vitro screen for spermatogenic toxicity.

The last two decades have seen an increasing search for in vitro models that can replace the use of animals for safety testing. We adapted the methods from a recent nonquantitative report of spermatogenesis occurring in ex vivo mouse testis explants and tried to develop them into a screening assay. The model consisted of small pieces of neonatal mouse testis (testis "chunks"), explanted and placed on pillars of agarose or chamber inserts, and cultured at the air-liquid interface. A peripheral torus-shaped zone in these explants would often contain tubules showing spermatogenesis, while the middle of each chunk was often necrotic, depending on the thickness of the tissue. The endpoint was histology: what proportion of tubules in the "permissive torus" actually contained healthy pachytene spermatocytes or spermatids? Extensive statistical modeling revealed that a useful predictive model required more than 60% of these tubules to show spermatogenesis. Separately, the logistics of running this as a predictive assay require that the controls consistently produce ≥ 60% tubules with pachytenes and round spermatids, and achieving this level of spermatogenesis reliably and consistently every week proved ultimately not possible. Extensive trials with various media additions and amendments proved incapable of maintaining the frequency of spermatogenic tubules at consistently ≥ 60%. Congruent with Schooler's "decline effect"; generally, the more often we ran these cultures, the worse the performance became. We hope that future efforts in this area may use our experience as a starting point on the way to a fully productive in vitro model of spermatogenesis.

Development of a squamous cell carcinoma mouse model for immunotoxicity testing.

An important component of safety assessment of new pharmaceuticals is evaluation of their potential to increase the risk of developing cancer in humans. The traditional 2-year rodent bioassay often is not feasible or scientifically applicable for evaluation of biotherapeutics. Additionally, it has poor predictive value for non-genotoxic immunosuppressive compounds. Thus, there is a need for alternative testing strategies. A novel 3-stage tumor model in syngeneic C3H/HeN mice was evaluated here to study the effects of immunosuppressive drugs on tumor promotion and progression in vivo. The model employed a skin squamous cell carcinoma cell line (SCC VII) due to the increased prevalence of squamous cell carcinoma (SCC) in humans associated with immunosuppression after transplants. Local invasion, colonization and tumor progression were evaluated. The validation set of immunosuppressive drugs included: Cyclosporin (CSA), cyclophosphamide (CTX), azathioprine, etanercept, abatacept and prednisone. Local invasion was evaluated by histological assessment as well as fluorescence trafficking from Qdot(®)-labeled tumor cells from the site of inoculation to the draining lymph node. Colonization was evaluated by lung colony counts following intravenous inoculation. Tumor progression was assessed by morphometric analysis of lesion area, angiogenesis and growth fraction of established metastatic neoplasia. Immunosuppressive drugs in the validation set yielded mixed results, including decreased progression. The methods and results described herein using an in vivo syngeneic mouse tumor model can provide insight about the assessment of immunosuppressive drugs in carcinogenicity risk assessment.

Inter-Method Performance Study of Tumor Volumetry Assessment on Computed Tomography Test-Retest Data.

RATIONALE AND OBJECTIVES: Tumor volume change has potential as a biomarker for diagnosis, therapy planning, and treatment response. Precision was evaluated and compared among semiautomated lung tumor volume measurement algorithms from clinical thoracic computed tomography data sets. The results inform approaches and testing requirements for establishing conformance with the Quantitative Imaging Biomarker Alliance (QIBA) Computed Tomography Volumetry Profile. MATERIALS AND METHODS: Industry and academic groups participated in a challenge study. Intra-algorithm repeatability and inter-algorithm reproducibility were estimated. Relative magnitudes of various sources of variability were estimated using a linear mixed effects model. Segmentation boundaries were compared to provide a basis on which to optimize algorithm performance for developers. RESULTS: Intra-algorithm repeatability ranged from 13% (best performing) to 100% (least performing), with most algorithms demonstrating improved repeatability as the tumor size increased. Inter-algorithm reproducibility was determined in three partitions and was found to be 58% for the four best performing groups, 70% for the set of groups meeting repeatability requirements, and 84% when all groups but the least performer were included. The best performing partition performed markedly better on tumors with equivalent diameters greater than 40 mm. Larger tumors benefitted by human editing but smaller tumors did not. One-fifth to one-half of the total variability came from sources independent of the algorithms. Segmentation boundaries differed substantially, not ony in overall volume but also in detail. CONCLUSIONS: Nine of the 12 participating algorithms pass precision requirements similar to what is indicated in the QIBA Profile, with the caveat that the present study was not designed to explicitly evaluate algorithm profile conformance. Change in tumor volume can be measured with confidence to within ±14% using any of these nine algorithms on tumor sizes greater than 10 mm. No partition of the algorithms was able to meet the QIBA requirements for interchangeability down to 10 mm, although the partition comprising best performing algorithms did meet this requirement for a tumor size of greater than approximately 40 mm.

STEP levels are unchanged in pre-frontal cortex and associative striatum in post-mortem human brain samples from subjects with schizophrenia, bipolar disorder and major depressive disorder.

Increased protein levels of striatal-enriched tyrosine phosphatase (STEP) have recently been reported in postmortem schizophrenic cortex. The present study sought to replicate this finding in a separate cohort of postmortem samples and to extend observations to striatum, including subjects with bipolar disorder and major depressive disorder in the analysis. No statistically significant changes between disease and control subjects were found in STEP mRNA or protein levels in dorsolateral prefrontal cortex or associative striatum. Although samples were matched for several covariates, postmortem interval correlated negatively with STEP protein levels, emphasizing the importance of including these analyses in postmortem studies.

Comparison of the Sysmex XT-2000iV and microscopic bone marrow differential counts in Wistar rats treated with cyclophosphamide, erythropoietin, or serial phlebotomy.

BACKGROUND: In a previous study, it was demonstrated that bone marrow analysis using the Sysmex XT-2000iV hematology analyzer produced differential counts in untreated rats that were comparable to microscopic differential counts. OBJECTIVE: The aim of this study was to modulate hematopoiesis in rats in vivo either through pharmacologic treatment or serial phlebotomy, and to determine whether the Sysmex XT-2000iV could accurately analyze bone marrow quantitative changes when compared with results obtained by microscopy. METHODS: Rats were treated once with 0, 5, 20, and 40 mg/kg cyclophosphamide (CP), 0, 50, 100 IU/kg erythropoietin (EPO) on 4 consecutive days, or serial phlebotomy of 1-2 mL of blood for 4 days. Modulation of hematopoietic populations in bone marrow was evaluated using the Sysmex XT-2000iV hematology analyzer, and compared with microscopic differential counts. RESULTS: Correlation coefficients between M:E ratios determined by Sysmex and the microscopic method were 0.94, 0.96, and 0.98 for CP, EPO, or serial phlebotomy treatments, respectively. Mean concordance correlation coefficients for M:E demonstrated method agreement of 0.63, 0.92, and 0.85 for the 3 treatments. Quantitative automated and microscopic bone marrow differential counts were within the expected 95% confidence intervals for CP, EPO or serial phlebotomy. CONCLUSIONS: The Sysmex XT-2000iV provides quantitative bone marrow differential counts of bone marrow cell series in rats with treatment-induced changes which are comparable to microscopic differential counts. Reliable automatic bone marrow differential counting allows increased throughput, sensitivity, reproducibility, and enhanced interpretation of bone marrow evaluation in rodent preclinical studies.

Validation of Sysmex XT-2000iV generated quantitative bone marrow differential counts in untreated Wistar rats.

BACKGROUND: Preclinical drug trials frequently require assessment of bone marrow toxicity in animals to evaluate hematopoietic safety. Since the gold standard, cytologic evaluation, is time consuming and requires highly trained individuals, automated methods remain intriguing. OBJECTIVE: The Sysmex XT-2000iV hematology analyzer allows user-developed customizable gating. This study was conducted to validate the gating of bone marrow cell populations in Sysmex cytograms from untreated rats. METHODS: B- and T-lymphocytes and myeloid cells were experimentally depleted from Charles River Wistar Han IGS (CRL: WI [Han]) rat whole bone marrow suspension using a magnetic cell sorting (MACS) method. The positively and negatively selected populations were used to verify select gates within the Sysmex cytogram. Intra- and inter-animal precision, comparability between right and left femur, as well as agreement with microscopic myelograms based on 500 counted cells, were determined. RESULTS: Intra-sample precision and right-to-left femur comparability confirmed that gating was reproducible and stable. In 50 tested rats, myeloid to erythroid ratios (M:E) were 1.32 ± 0.33 in males and 1.38 ± 0.29 in females by Sysmex compared to 1.36 ± 0.32 in males and 1.42 ± 0.32 in females by microscopic evaluations. Bland-Altman differences between methods was ≤ ± 0.35 units for M:E, ≤ 5.4% for maturing myeloid cells, ≤ 3.4% for proliferating myeloid cells, ≤ 6.0% for maturing myeloid cells, ≤ 3.4% for proliferating myeloid cells, and ≤ 4.1% for lymphocytes. CONCLUSIONS: In untreated control Charles River Wistar Han IGS (CRL: WI [Han]) rats, the Sysmex XT-2000iV produced an automated M:E and 5-part differential count equivalent to microscopic differential counts.

Three-dimensional lung tumor microenvironment modulates therapeutic compound responsiveness in vitro--implication for drug development.

Three-dimensional (3D) cell culture is gaining acceptance in response to the need for cellular models that better mimic physiologic tissues. Spheroids are one such 3D model where clusters of cells will undergo self-assembly to form viable, 3D tumor-like structures. However, to date little is known about how spheroid biology compares to that of the more traditional and widely utilized 2D monolayer cultures. Therefore, the goal of this study was to characterize the phenotypic and functional differences between lung tumor cells grown as 2D monolayer cultures, versus cells grown as 3D spheroids. Eight lung tumor cell lines, displaying varying levels of epidermal growth factor receptor (EGFR) and cMET protein expression, were used to develop a 3D spheroid cell culture model using low attachment U-bottom plates. The 3D spheroids were compared with cells grown in monolayer for 1) EGFR and cMET receptor expression, as determined by flow cytometry, 2) EGFR and cMET phosphorylation by MSD assay, and 3) cell proliferation in response to epidermal growth factor (EGF) and hepatocyte growth factor (HGF). In addition, drug responsiveness to EGFR and cMET inhibitors (Erlotinib, Crizotinib, Cetuximab [Erbitux] and Onartuzumab [MetMab]) was evaluated by measuring the extent of cell proliferation and migration. Data showed that EGFR and cMET expression is reduced at day four of untreated spheroid culture compared to monolayer. Basal phosphorylation of EGFR and cMET was higher in spheroids compared to monolayer cultures. Spheroids showed reduced EGFR and cMET phosphorylation when stimulated with ligand compared to 2D cultures. Spheroids showed an altered cell proliferation response to HGF, as well as to EGFR and cMET inhibitors, compared to monolayer cultures. Finally, spheroid cultures showed exceptional utility in a cell migration assay. Overall, the 3D spheroid culture changed the cellular response to drugs and growth factors and may more accurately mimic the natural tumor microenvironment.

The inhibin B response in male rats treated with two drug candidates.

BACKGROUND: Serum Inhibin B was measured in two studies of known testis-toxic drug candidates. METHODS AND RESULTS: Study 1 was for a compound for Hepatitis C, and utilized a 10-week dosing period, followed by mating and necropsy of half of each group, and then a 12-week recovery period for the remaining animals. At the postmating necropsy, 6 of 15 high-dose males had testis lesions; Inhibin B was significantly reduced in all animals in that group. The mid-dose group had no lesions but significantly reduced serum Inhibin B. At recovery, 9 of 15 high-dose males showed damage in testes; serum Inhibin B levels were not different from controls. Inhibin B appeared to both overreport and underreport testis damage in Study 1. Study 2 was an acute pathogenesis study for an antibacterial compound, using control and two dose levels and multiple time points (days 5, 8, 15, 22, and then untreated until day 71). At each time point blood was sampled from all remaining rats and five/group were killed for histologic evaluation. The low-dose group had minimal to moderate lesions, while serum Inhibin B was never changed. The high-dose animals progressed quickly from minimal lesions to being broadly and moderately affected; serum Inhibin B levels were reduced at days 8 and 15 only. In Study 2, Inhibin B appeared less sensitive than histology, except at the extremes of testis damage, when Inhibin B was routinely low. CONCLUSION: We conclude that in these two studies there was a poor correlation between changes in serum levels of Inhibin B and testis histopathology.

Multinomial modeling and an evaluation of common data-mining algorithms for identifying signals of disproportionate reporting in pharmacovigilance databases.

MOTIVATION: A principal objective of pharmacovigilance is to detect adverse drug reactions that are unknown or novel in terms of their clinical severity or frequency. One method is through inspection of spontaneous reporting system databases, which consist of millions of reports of patients experiencing adverse effects while taking one or more drugs. For such large databases, there is an increasing need for quantitative and automated screening tools to assist drug safety professionals in identifying drug-event combinations (DECs) worthy of further investigation. Existing algorithms can effectively identify problematic DECs when the frequencies are high. However these algorithms perform differently for low-frequency DECs. RESULTS: In this work, we provide a method based on the multinomial distribution that identifies signals of disproportionate reporting, especially for low-frequency combinations. In addition, we comprehensively compare the performance of commonly used algorithms with the new approach. Simulation results demonstrate the advantages of the proposed method, and analysis of the Adverse Event Reporting System data shows that the proposed method can help detect interesting signals. Furthermore, we suggest that these methods be used to identify DECs that occur significantly less frequently than expected, thus identifying potential alternative indications for these drugs. We provide an empirical example that demonstrates the importance of exploring underexpected DECs. AVAILABILITY: Code to implement the proposed method is available in R on request from the corresponding authors. CONTACT: kjell@arboranalytics.com or Mark.M.Gosink@Pfizer.com SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Statistical analysis and compound selection of combinatorial libraries for soluble epoxide hydrolase.

Inhibitors of soluble epoxide hydrolase (sEH) have been extensively pursued as antihypertensive therapies as well as potential treatment for other cardiovascular dysfunctions and prevention of renal damage. In this study we report quantitative structure-activity relationship (QSAR) models for 1223 structurally diverse sEH inhibitors produced by combinatorial library design and synthesis. Daylight fingerprints, MOE 2D and DragonX descriptors were generated for QSAR modeling approaches. Using these descriptors, a number of statistical models were trained and validated. Of these methods, gradient boosting machines (GBM), partial least-squares (PLS), and Cubist methods demonstrated the best performance on training and test set validation in terms of their leave-group-out cross-validated (LGO-CV) Q(2) and correlation coefficient R(2) (Q(2)(GBM-training) = 0.79, R(2)(GBM-test) = 0.81; Q(2)(PLS-training) = 0.75, R(2)(PLS-test) = 0.75; Q(2)(Cubist-training) = 0.91, R(2)(Cubist-test) = 0.78). A final model was constructed using the consensus approach of the three individual models and showed robust statistics and prediction of the external validation set. The Gaussian process modified sequential elimination of level combinations (G-SELC) method was then used to expand the chemical space beyond what has been explored by combinatorial synthesis. This approach identified 50 new compounds that are structurally diverse and potentially desirable for sEH inhibition based on prior knowledge. The activities of the suggested compounds were then predicted by the consensus QSAR model, and the results supported that the compounds were more likely to exist in the active parts of the chemical space. This study illustrates that the balanced approach by G-SELC could provide a general method for combinatorial library design, to effectively identify promising compounds to be created in the laboratory.

Not a walk in the park: the ECVAM whole embryo culture model challenged with pharmaceuticals and attempted improvements with random forest design.

BACKGROUND: The European Committee for the Validation of Alternative Methods (ECVAM) supported the development of a linear discriminant embryotoxicity prediction model founded on rat whole embryo culture (Piersma et al. (2004). Altern Lab Anim 32:275­307). Our goals were to (1) assess the accuracy of this model with pharmaceuticals, and (2) to use the data to develop a more accurate prediction model. METHODS: Sixty-one chemicals of known in vivo activity were tested. They were part of the ECVAM validation set (N513), commercially available pharmaceuticals (N531), and Pfizer chemicals that did not reach the market, but for which developmental toxicity data were available (N517). They were tested according to the ECVAM procedures. Fifty-seven of these chemicals were used for Random Forest modeling to develop an alternate model with the goal of using surrogate endpoints for simplified assessments and to improve the predictivity of the model. RESULTS: Using part of the ECVAM chemical test set, the ECVAM prediction model was 77% accurate. This approximated what was reported in the validation study (80%; Piersma et al. (2004). Altern Lab Anim 32:275­307). However, when confronted with novel chemicals, the accuracy of the linear discriminant model dropped to 56%. In an attempt to improve this performance, we used a Random Forest model that provided rankings and confidence estimates. Although the model used simpler endpoints, its performance was no better than the ECVAM linear discriminant model. CONCLUSIONS: This study confirms previous concerns about the applicability of the ECVAM prediction model to a more diverse chemical set, and underscores the challenges associated with developing embryotoxicity prediction models.

The ensemble bridge algorithm: a new modeling tool for drug discovery problems.

Ensemble algorithms have been historically categorized into two separate paradigms, boosting and random forests, which differ significantly in the way each ensemble is constructed. Boosting algorithms represent one extreme, where an iterative greedy optimization strategy, weak learners (e.g., small classification trees), and stage weights are employed to target difficult-to-classify regions in the training space. On the other extreme, random forests rely on randomly selected features and complex learners (learners that exhibit low bias, e.g., large regression trees) to classify well over the entire training data. Because the approach is not targeting the next learner for inclusion, it tends to provide a natural robustness to noisy labels. In this work, we introduce the ensemble bridge algorithm, which is capable of transitioning between boosting and random forests using a regularization parameter nu in [0,1]. Because the ensemble bridge algorithm is a compromise between the greedy nature of boosting and the randomness present in random forests, it yields robust performance in the presence of a noisy response and superior performance in the presence of a clean response. Often, drug discovery data (e.g., computational chemistry data) have varying levels of noise. Hence, this method enables a practitioner to employ a single method to evaluate ensemble performance. The method's robustness is verified across a variety of data sets where the algorithm repeatedly yields better performance than either boosting or random forests alone. Finally, we provide diagnostic tools for the new algorithm, including a measure of variable importance and an observational clustering tool.