A machine learning case study to predict rare clinical event of interest: imbalanced data, interpretability, and practical considerations.

Accurate prediction of a rare and clinically important event following study treatment has been crucial in drug development. For instance, the rarity of an adverse event is often commensurate with the seriousness of medical consequences, and delayed detection of the rare adverse event can pose significant or even life-threatening health risks to patients. In this machine learning case study, we demonstrate with an example originated from a real clinical trial setting how to define and solve the rare clinical event prediction problem using machine learning in pharmaceutical industry. The unique contributions of this work include the proposal of a six-step investigation framework that facilitates the communication with non-technical stakeholders and the interpretation of the model performance in terms of practical consequences in the context of patient screenings for conducting a future clinical trial. In terms of machine learning methodology, for data splitting into the training and test sets, we adapt the rare-event stratified split approach (from scikit-learn) to further account for group splitting for multiple records of a patient simultaneously. To handle imbalanced data due to rare events in model training, the cost-sensitive learning approach is employed to give more weights to the minor class and the metrics precision together with recall are used to capture prediction performance instead of the raw accuracy rate. Finally, we demonstrate how to apply the state-of-the-art SHAP values to identify important risk factors to improve model interpretability.

Enrollment forecast for clinical trials at the portfolio planning phase based on site-level historical data.

An accurate forecast of a clinical trial enrollment timeline at the planning phase is of great importance to both corporate strategic planning and trial operational excellence. The naive approach often calculates an average enrollment rate from historical data and generates an inaccurate prediction based on a linear trend with the average rate. Under the traditional framework of a Poisson-Gamma model, site activation delays are often modeled with either fixed initiation time or a simple random distribution while incorporating the user-provided site planning information to achieve good forecast accuracy. However, such user-provided information is not available at the early portfolio planning stage. We present a novel statistical approach based on generalized linear mixed-effects models and the use of non-homogeneous Poisson processes through the Bayesian framework to model the country initiation, site activation, and subject enrollment sequentially in a systematic fashion. We validate the performance of our proposed enrollment modeling framework based on a set of 25 preselected studies from four therapeutic areas. Our modeling framework shows a substantial improvement in prediction accuracy in comparison to the traditional statistical approach. Furthermore, we show that our modeling and simulation approach calibrates the data variability appropriately and gives correct coverage rates for prediction intervals of various nominal levels. Finally, we demonstrate the use of our approach to generate the predicted enrollment curves through time with confidence bands overlaid.

Quantifying Nocturnal Scratch in Atopic Dermatitis: A Machine Learning Approach Using Digital Wrist Actigraphy.

Nocturnal scratching substantially impairs the quality of life in individuals with skin conditions such as atopic dermatitis (AD). Current clinical measurements of scratch rely on patient-reported outcomes (PROs) on itch over the last 24 h. Such measurements lack objectivity and sensitivity. Digital health technologies (DHTs), such as wearable sensors, have been widely used to capture behaviors in clinical and real-world settings. In this work, we develop and validate a machine learning algorithm using wrist-wearing actigraphy that could objectively quantify nocturnal scratching events, therefore facilitating accurate assessment of disease progression, treatment effectiveness, and overall quality of life in AD patients. A total of seven subjects were enrolled in a study to generate data overnight in an inpatient setting. Several machine learning models were developed, and their performance was compared. Results demonstrated that the best-performing model achieved the F1 score of 0.45 on the test set, accompanied by a precision of 0.44 and a recall of 0.46. Upon satisfactory performance with an expanded subject pool, our automatic scratch detection algorithm holds the potential for objectively assessing sleep quality and disease state in AD patients. This advancement promises to inform and refine therapeutic strategies for individuals with AD.

Enrollment Forecast for Clinical Trials at the Planning Phase with Study-Level Historical Data.

Given progressive developments and demands on clinical trials, accurate enrollment timeline forecasting is increasingly crucial for both strategic decision-making and trial execution excellence. Naïve approach assumes flat rates on enrollment using average of historical data, while traditional statistical approach applies simple Poisson-Gamma model using time-invariant rates for site activation and subject recruitment. Both of them are lack of non-trivial factors such as time and location. We propose a novel two-segment statistical approach based on Quasi-Poisson regression for subject accrual rate and Poisson process for subject enrollment and site activation. The input study-level data are publicly accessible and it can be integrated with historical study data from user's organization to prospectively predict enrollment timeline. The new framework is neat and accurate compared to preceding works. We validate the performance of our proposed enrollment model and compare the results with other frameworks on 7 curated studies.

Deep Learning-Based Psoriasis Assessment: Harnessing Clinical Trial Imaging for Accurate Psoriasis Area Severity Index Prediction.

Introduction: Image-based machine learning holds great promise for facilitating clinical care; however, the datasets often used for model training differ from the interventional clinical trial-based findings frequently used to inform treatment guidelines. Here, we draw on longitudinal imaging of psoriasis patients undergoing treatment in the Ultima 2 clinical trial (NCT02684357), including 2,700 body images with psoriasis area severity index (PASI) annotations by uniformly trained dermatologists. Methods: An image-processing workflow integrating clinical photos of multiple body regions into one model pipeline was developed, which we refer to as the "One-Step PASI" framework due to its simultaneous body detection, lesion detection, and lesion severity classification. Group-stratified cross-validation was performed with 145 deep convolutional neural network models combined in an ensemble learning architecture. Results: The highest-performing model demonstrated a mean absolute error of 3.3, Lin's concordance correlation coefficient of 0.86, and Pearson correlation coefficient of 0.90 across a wide range of PASI scores comprising disease classifications of clear skin, mild, and moderate-to-severe disease. Within-person, time-series analysis of model performance demonstrated that PASI predictions closely tracked the trajectory of physician scores from severe to clear skin without systematically over- or underestimating PASI scores or percent changes from baseline. Conclusion: This study demonstrates the potential of image processing and deep learning to translate otherwise inaccessible clinical trial data into accurate, extensible machine learning models to assess therapeutic efficacy.

Dehydration of Ions in Voltage-Gated Carbon Nanopores Observed by in Situ NMR.

Ion transport through nanochannels is of fundamental importance in voltage-gated protein ion channels and energy storage devices. Direct microscopic observations are critical for understanding the intricacy of ionic processes in nanoconfinement. Here we report an in situ nuclear magnetic resonance study of ion hydration in voltage-gated carbon nanopores. Nucleus-independent chemical shift was employed to monitor the ionic processes of NaF aqueous electrolyte in nanopores of carbon supercapacitors. The state of ion hydration was revealed by the chemical shift, which is sensitive to the hydration number. A large energy barrier was observed for ions to enter nanopores smaller than the hydrated ion size. Increasing the gating voltage above 0.4 V overcomes this barrier and brings F(-) into the nanopores without dehydration. Partial dehydration of F(-) occurs only at gating voltage above 0.7 V. No dehydration was observed for Na(+) cations, in agreement with their strong ion hydration.

Electroneutrality breakdown and specific ion effects in nanoconfined aqueous electrolytes observed by NMR.

Ion distribution in aqueous electrolytes near the interface plays a critical role in electrochemical, biological and colloidal systems, and is expected to be particularly significant inside nanoconfined regions. Electroneutrality of the total charge inside nanoconfined regions is commonly assumed a priori in solving ion distribution of aqueous electrolytes nanoconfined by uncharged hydrophobic surfaces with no direct experimental validation. Here, we use a quantitative nuclear magnetic resonance approach to investigate the properties of aqueous electrolytes nanoconfined in graphitic-like nanoporous carbon. Substantial electroneutrality breakdown in nanoconfined regions and very asymmetric responses of cations and anions to the charging of nanoconfining surfaces are observed. The electroneutrality breakdown is shown to depend strongly on the propensity of anions towards the water-carbon interface and such ion-specific response follows, generally, the anion ranking of the Hofmeister series. The experimental observations are further supported by numerical evaluation using the generalized Poisson-Boltzmann equation.