Evaluating robotic-assisted partial nephrectomy surgeons with fully convolutional segmentation and multi-task attention networks.

We use machine learning to evaluate surgical skill from videos during the tumor resection and renography steps of a robotic assisted partial nephrectomy (RAPN). This expands previous work using synthetic tissue to include actual surgeries. We investigate cascaded neural networks for predicting surgical proficiency scores (OSATS and GEARS) from RAPN videos recorded from the DaVinci system. The semantic segmentation task generates a mask and tracks the various surgical instruments. The movements from the instruments found via semantic segmentation are processed by a scoring network that regresses (predicts) GEARS and OSATS scoring for each subcategory. Overall, the model performs well for many subcategories such as force sensitivity and knowledge of instruments of GEARS and OSATS scoring, but can suffer from false positives and negatives that would not be expected of human raters. This is mainly attributed to limited training data variability and sparsity.

Towards automated molecular detection through simulated generation of CMOS-based rotational spectroscopy.

The use of CMOS sensors for rotational spectroscopy is a promising, but challenging avenue for low-cost gas sensing and molecular identification. A main challenge in this approach is that practical CMOS spectroscopy samples contain various different noise sources that reduce the effectiveness of matching techniques for molecular identification with rotational spectroscopy. To help solve this challenge, we develop a software application tool that can demonstrate the feasibility and reliability of detection with CMOS sensor samples. Specifically, the tool characterizes the types of noise in CMOS sample collection and synthesizes spectroscopy files based upon existing databases of rotational spectroscopy samples gathered from other sensors. We use the software to create a large database of plausible CMOS-generated sample files of gases. This dataset is used to help evaluate spectral matching algorithms used in gas sensing and molecular identification applications. We evaluate these traditional methods on the synthesized dataset and discuss how peak finding and spectral matching algorithms can be altered to accommodate the noise sources present in CMOS sample collection.

LAST: Latent Space-Assisted Adaptive Sampling for Protein Trajectories.

Molecular dynamics (MD) simulation is widely used to study protein conformations and dynamics. However, conventional simulation suffers from being trapped in some local energy minima that are hard to escape. Thus, most of the computational time is spent sampling in the already visited regions. This leads to an inefficient sampling process and further hinders the exploration of protein movements in affordable simulation time. The advancement of deep learning provides new opportunities for protein sampling. Variational autoencoders are a class of deep learning models to learn a low-dimensional representation (referred to as the latent space) that can capture the key features of the input data. Based on this characteristic, we proposed a new adaptive sampling method, latent space-assisted adaptive sampling for protein trajectories (LAST), to accelerate the exploration of protein conformational space. This method comprises cycles of (i) variational autoencoder training, (ii) seed structure selection on the latent space, and (iii) conformational sampling through additional MD simulations. The proposed approach is validated through the sampling of four structures of two protein systems: two metastable states of Escherichia coli adenosine kinase (ADK) and two native states of Vivid (VVD). In all four conformations, seed structures were shown to lie on the boundary of conformation distributions. Moreover, large conformational changes were observed in a shorter simulation time when compared with structural dissimilarity sampling (SDS) and conventional MD (cMD) simulations in both systems. In metastable ADK simulations, LAST explored two transition paths toward two stable states, while SDS explored only one and cMD neither. In VVD light state simulations, LAST was three times faster than cMD simulation with a similar conformational space. Overall, LAST is comparable to SDS and is a promising tool in adaptive sampling. The LAST method is publicly available at https://github.com/smu-tao-group/LAST to facilitate related research.

Smartphone camera oximetry in an induced hypoxemia study.

Hypoxemia, a medical condition that occurs when the blood is not carrying enough oxygen to adequately supply the tissues, is a leading indicator for dangerous complications of respiratory diseases like asthma, COPD, and COVID-19. While purpose-built pulse oximeters can provide accurate blood-oxygen saturation (SpO2) readings that allow for diagnosis of hypoxemia, enabling this capability in unmodified smartphone cameras via a software update could give more people access to important information about their health. Towards this goal, we performed the first clinical development validation on a smartphone camera-based SpO2 sensing system using a varied fraction of inspired oxygen (FiO2) protocol, creating a clinically relevant validation dataset for solely smartphone-based contact PPG methods on a wider range of SpO2 values (70-100%) than prior studies (85-100%). We built a deep learning model using this data to demonstrate an overall MAE = 5.00% SpO2 while identifying positive cases of low SpO2 < 90% with 81% sensitivity and 79% specificity. We also provide the data in open-source format, so that others may build on this work.

Evaluating robotic-assisted surgery training videos with multi-task convolutional neural networks.

We seek to understand if an automated algorithm can replace human scoring of surgical trainees performing the urethrovesical anastomosis in radical prostatectomy with synthetic tissue. Specifically, we investigate neural networks for predicting the surgical proficiency score (GEARS score) from video clips. We evaluate videos of surgeons performing the urethral anastomosis using synthetic tissue. The algorithm tracks surgical instrument locations from video, saving the positions of key points on the instruments over time. These positional features are used to train a multi-task convolutional network to infer each sub-category of the GEARS score to determine the proficiency level of trainees. Experimental results demonstrate that the proposed method achieves good performance with scores matching manual inspection in 86.1% of all GEARS sub-categories. Furthermore, the model can detect the difference between proficiency (novice to expert) in 83.3% of videos. Evaluation of GEARS sub-categories with artificial neural networks is possible for novice and intermediate surgeons, but additional research is needed to understand if expert surgeons can be evaluated with a similar automated system.

Explore Protein Conformational Space With Variational Autoencoder.

Molecular dynamics (MD) simulations have been actively used in the study of protein structure and function. However, extensive sampling in the protein conformational space requires large computational resources and takes a prohibitive amount of time. In this study, we demonstrated that variational autoencoders (VAEs), a type of deep learning model, can be employed to explore the conformational space of a protein through MD simulations. VAEs are shown to be superior to autoencoders (AEs) through a benchmark study, with low deviation between the training and decoded conformations. Moreover, we show that the learned latent space in the VAE can be used to generate unsampled protein conformations. Additional simulations starting from these generated conformations accelerated the sampling process and explored hidden spaces in the conformational landscape.

Generative adversarial networks for transition state geometry prediction.

This work introduces a novel application of generative adversarial networks (GANs) for the prediction of starting geometries in transition state (TS) searches based on the geometries of reactants and products. The multi-dimensional potential energy space of a chemical reaction often complicates the location of a starting TS geometry, leading to the correct TS combining reactants and products in question. The proposed TS-GAN efficiently maps the space between reactants and products and generates reliable TS guess geometries, and it can be easily combined with any quantum chemical software package performing geometry optimizations. The TS-GAN was trained and applied to generate TS guess structures for typical chemical reactions, such as hydrogen migration, isomerization, and transition metal-catalyzed reactions. The performance of the TS-GAN was directly compared to that of classical approaches, proving its high accuracy and efficiency. The current TS-GAN can be extended to any dataset that contains sufficient chemical reactions for training. The software is freely available for training, experimentation, and prediction at https://github.com/ekraka/TS-GAN.

Deep Learning-Based Ligand Design Using Shared Latent Implicit Fingerprints from Collaborative Filtering.

In their previous work, Srinivas et al. [ J. Cheminf. 2018, 10, 56] have shown that implicit fingerprints capture ligands and proteins in a shared latent space, typically for the purposes of virtual screening with collaborative filtering models applied on known bioactivity data. In this work, we extend these implicit fingerprints/descriptors using deep learning techniques to translate latent descriptors into discrete representations of molecules (SMILES), without explicitly optimizing for chemical properties. This allows the design of new compounds based upon the latent representation of nearby proteins, thereby encoding druglike properties including binding affinities to known proteins. The implicit descriptor method does not require any fingerprint similarity search, which makes the method free of any bias arising from the empirical nature of the fingerprint models [Srinivas, R.; J. Cheminf. 2018, 10, 56]. We evaluate the properties of the potentially novel drugs generated by our approach using physical properties of druglike molecules and chemical complexity. Additionally, we analyze the reliability of the biological activity of the new compounds generated using this method by employing models of protein-ligand interaction, which assists in assessing the potential binding affinity of the designed compounds. We find that the generated compounds exhibit properties of chemically feasible compounds and are predicted to be excellent binders to known proteins. Furthermore, we also analyze the diversity of compounds created using the Tanimoto distance and conclude that there is a wide diversity in the generated compounds.

SSnet: A Deep Learning Approach for Protein-Ligand Interaction Prediction.

Computational prediction of Protein-Ligand Interaction (PLI) is an important step in the modern drug discovery pipeline as it mitigates the cost, time, and resources required to screen novel therapeutics. Deep Neural Networks (DNN) have recently shown excellent performance in PLI prediction. However, the performance is highly dependent on protein and ligand features utilized for the DNN model. Moreover, in current models, the deciphering of how protein features determine the underlying principles that govern PLI is not trivial. In this work, we developed a DNN framework named SSnet that utilizes secondary structure information of proteins extracted as the curvature and torsion of the protein backbone to predict PLI. We demonstrate the performance of SSnet by comparing against a variety of currently popular machine and non-Machine Learning (ML) models using various metrics. We visualize the intermediate layers of SSnet to show a potential latent space for proteins, in particular to extract structural elements in a protein that the model finds influential for ligand binding, which is one of the key features of SSnet. We observed in our study that SSnet learns information about locations in a protein where a ligand can bind, including binding sites, allosteric sites and cryptic sites, regardless of the conformation used. We further observed that SSnet is not biased to any specific molecular interaction and extracts the protein fold information critical for PLI prediction. Our work forms an important gateway to the general exploration of secondary structure-based Deep Learning (DL), which is not just confined to protein-ligand interactions, and as such will have a large impact on protein research, while being readily accessible for de novo drug designers as a standalone package.

Measuring Oxygen Saturation With Smartphone Cameras Using Convolutional Neural Networks.

Arterial oxygen saturation ([Formula: see text]) is an indicator of how much oxygen is carried by hemoglobin in the blood. Having enough oxygen is vital for the functioning of cells in the human body. Measurement of [Formula: see text] is typically estimated with a pulse oximeter, but recent works have investigated how smartphone cameras can be used to infer [Formula: see text]. In this paper, we propose methods for the measurement of [Formula: see text] with a smartphone using convolutional neural networks and preprocessing steps to better guard against motion artifacts. To evaluate this methodology, we conducted a breath-holding study involving 39 participants. We compare the results using two different mobile phones. We compare our model with the ratio-of-ratios model that is widely used in pulse oximeter applications, showing that our system has significantly lower mean absolute error (2.02%) than a medical pulse oximeter.

Implicit-descriptor ligand-based virtual screening by means of collaborative filtering.

Current ligand-based machine learning methods in virtual screening rely heavily on molecular fingerprinting for preprocessing, i.e., explicit description of ligands' structural and physicochemical properties in a vectorized form. Of particular importance to current methods are the extent to which molecular fingerprints describe a particular ligand and what metric sufficiently captures similarity among ligands. In this work, we propose and evaluate methods that do not require explicit feature vectorization through fingerprinting, but, instead, provide implicit descriptors based only on other known assays. Our methods are based upon well known collaborative filtering algorithms used in recommendation systems. Our implicit descriptor method does not require any fingerprint similarity search, which makes the method free of the bias arising from the empirical nature of the fingerprint models. We show that implicit methods significantly outperform traditional machine learning methods, and the main strengths of implicit methods are their resilience to target-ligand sparsity and high potential for spotting promiscuous ligands.

Use of a Smartphone App to Assess Neonatal Jaundice.

BACKGROUND: The assessment of jaundice in outpatient neonates is problematic. Visual assessment is inaccurate, and more exact methodologies are cumbersome and/or expensive. Our goal in this study was to assess the accuracy of a technology based on the analysis of digital images of newborns obtained using a smartphone application called BiliCam. METHODS: Paired BiliCam images and total serum bilirubin (TSB) levels were obtained in a diverse sample of newborns (<7 days old) at 7 sites across the United States. By using specialized software, data on color values in the images ("features") were extracted. Machine learning and regression analysis techniques were used to identify features for inclusion in models to predict an estimated bilirubin level for each newborn. The correlation between estimated bilirubin levels and TSB levels was calculated. In addition, the sensitivity and specificity of the estimated bilirubin levels in identifying newborns with high TSB levels were calculated by using 2 recommended decision rules for jaundice screening. RESULTS: Estimated bilirubin levels were calculated and compared with TSB levels in a diverse sample of 530 newborns (20.8% African American, 26.3% Hispanic, and 21.2% Asian American). The overall correlation was 0.91, and correlations among white, African American, Hispanic, and Asian American newborns were 0.92, 0.90, 0.91, and 0.88, respectively. The sensitivities of BiliCam in identifying newborns with high TSB levels were 84.6% and 100%, respectively, by using 2 decision rules; specificities were 75.1% and 76.4%, respectively. CONCLUSIONS: BiliCam provided accurate estimates of TSB values, demonstrating that an inexpensive technology that uses commodity smartphones could be used to effectively screen newborns for jaundice.