Development and Clinical Evaluation of an Artificial Intelligence Support Tool for Improving Telemedicine Photo Quality.

Importance: Telemedicine use accelerated during the COVID-19 pandemic, and skin conditions were a common use case. However, many images submitted may be of insufficient quality for making a clinical determination. Objective: To determine whether an artificial intelligence (AI) decision support tool, a machine learning algorithm, could improve the quality of images submitted for telemedicine by providing real-time feedback and explanations to patients. Design, Setting, and Participants: This quality improvement study with an AI performance component and single-arm clinical pilot study component was conducted from March 2020 to October 2021. After training, the AI decision support tool was tested on 357 retrospectively collected telemedicine images from Stanford telemedicine from March 2020 to June 2021. Subsequently, a single-arm clinical pilot study was conducted to assess feasibility with 98 patients in the Stanford Department of Dermatology across 2 clinical sites from July 2021 to October 2021. For the clinical pilot study, inclusion criteria for patients included being adults (aged ≥18 years), presenting to clinic for a skin condition, and being able to photograph their own skin with a smartphone. Interventions: During the clinical pilot study, patients were given a handheld smartphone device with a machine learning algorithm interface loaded and were asked to take images of any lesions of concern. Patients were able to review and retake photos prior to submitting, so each submitted photo met the patient's assumed standard of clinical acceptability. A machine learning algorithm then gave the patient feedback on whether the image was acceptable. If the image was rejected, the patient was provided a reason by the AI decision support tool and allowed to retake the photos. Main Outcomes and Measures: The main outcome of the retrospective image analysis was the receiver operator curve area under the curve (ROC-AUC). The main outcome of the clinical pilot study was the image quality difference between the baseline images and the images approved by AI decision support. Results: Of the 98 patients included, the mean (SD) age was 49.8 (17.6) years, and 50 (51%) of the patients were male. On retrospective telemedicine images, the machine learning algorithm effectively identified poor-quality images (ROC-AUC of 0.78) and the reason for poor quality (blurry ROC-AUC of 0.84; lighting issues ROC-AUC of 0.70). The performance was consistent across age and sex. In the clinical pilot study, patient use of the machine learning algorithm was associated with improved image quality. An AI algorithm was associated with reduction in the number of patients with a poor-quality image by 68.0%. Conclusions and Relevance: In this quality improvement study, patients use of the AI decision support with a machine learning algorithm was associated with improved quality of skin disease photographs submitted for telemedicine use.

Federated benchmarking of medical artificial intelligence with MedPerf.

Medical artificial intelligence (AI) has tremendous potential to advance healthcare by supporting and contributing to the evidence-based practice of medicine, personalizing patient treatment, reducing costs, and improving both healthcare provider and patient experience. Unlocking this potential requires systematic, quantitative evaluation of the performance of medical AI models on large-scale, heterogeneous data capturing diverse patient populations. Here, to meet this need, we introduce MedPerf, an open platform for benchmarking AI models in the medical domain. MedPerf focuses on enabling federated evaluation of AI models, by securely distributing them to different facilities, such as healthcare organizations. This process of bringing the model to the data empowers each facility to assess and verify the performance of AI models in an efficient and human-supervised process, while prioritizing privacy. We describe the current challenges healthcare and AI communities face, the need for an open platform, the design philosophy of MedPerf, its current implementation status and real-world deployment, our roadmap and, importantly, the use of MedPerf with multiple international institutions within cloud-based technology and on-premises scenarios. Finally, we welcome new contributions by researchers and organizations to further strengthen MedPerf as an open benchmarking platform.

AI-enabled in silico immunohistochemical characterization for Alzheimer's disease.

We develop a deep learning approach, in silico immunohistochemistry (IHC), which takes routinely collected histochemical-stained samples as input and computationally generates virtual IHC slide images. We apply in silico IHC to Alzheimer's disease samples, where several hallmark changes are conventionally identified using IHC staining across many regions of the brain. In silico IHC computationally identifies neurofibrillary tangles, ß-amyloid plaques, and neuritic plaques at a high spatial resolution directly from the histochemical images, with areas under the receiver operating characteristic curve of between 0.88 and 0.92. In silico IHC learns to identify subtle cellular morphologies associated with these lesions and can generate in silico IHC slides that capture key features of the actual IHC.

Integrating spatial gene expression and breast tumour morphology via deep learning.

Spatial transcriptomics allows for the measurement of RNA abundance at a high spatial resolution, making it possible to systematically link the morphology of cellular neighbourhoods and spatially localized gene expression. Here, we report the development of a deep learning algorithm for the prediction of local gene expression from haematoxylin-and-eosin-stained histopathology images using a new dataset of 30,612 spatially resolved gene expression data matched to histopathology images from 23 patients with breast cancer. We identified over 100 genes, including known breast cancer biomarkers of intratumoral heterogeneity and the co-localization of tumour growth and immune activation, the expression of which can be predicted from the histopathology images at a resolution of 100 µm. We also show that the algorithm generalizes well to The Cancer Genome Atlas and to other breast cancer gene expression datasets without the need for re-training. Predicting the spatially resolved transcriptome of a tissue directly from tissue images may enable image-based screening for molecular biomarkers with spatial variation.

Deep learning interpretation of echocardiograms.

Echocardiography uses ultrasound technology to capture high temporal and spatial resolution images of the heart and surrounding structures, and is the most common imaging modality in cardiovascular medicine. Using convolutional neural networks on a large new dataset, we show that deep learning applied to echocardiography can identify local cardiac structures, estimate cardiac function, and predict systemic phenotypes that modify cardiovascular risk but not readily identifiable to human interpretation. Our deep learning model, EchoNet, accurately identified the presence of pacemaker leads (AUC = 0.89), enlarged left atrium (AUC = 0.86), left ventricular hypertrophy (AUC = 0.75), left ventricular end systolic and diastolic volumes ( R 2 = 0.74 and R 2 = 0.70), and ejection fraction ( R 2 = 0.50), as well as predicted systemic phenotypes of age ( R 2 = 0.46), sex (AUC = 0.88), weight ( R 2 = 0.56), and height ( R 2 = 0.33). Interpretation analysis validates that EchoNet shows appropriate attention to key cardiac structures when performing human-explainable tasks and highlights hypothesis-generating regions of interest when predicting systemic phenotypes difficult for human interpretation. Machine learning on echocardiography images can streamline repetitive tasks in the clinical workflow, provide preliminary interpretation in areas with insufficient qualified cardiologists, and predict phenotypes challenging for human evaluation.

A primer on deep learning in genomics.

Deep learning methods are a class of machine learning techniques capable of identifying highly complex patterns in large datasets. Here, we provide a perspective and primer on deep learning applications for genome analysis. We discuss successful applications in the fields of regulatory genomics, variant calling and pathogenicity scores. We include general guidance for how to effectively use deep learning methods as well as a practical guide to tools and resources. This primer is accompanied by an interactive online tutorial.

3D-Printed Gastric Resident Electronics.

Long-term implantation of biomedical electronics into the human body enables advanced diagnostic and therapeutic functionalities. However, most long-term resident electronics devices require invasive procedures for implantation as well as a specialized receiver for communication. Here, a gastric resident electronic (GRE) system that leverages the anatomical space offered by the gastric environment to enable residence of an orally delivered platform of such devices within the human body is presented. The GRE is capable of directly interfacing with portable consumer personal electronics through Bluetooth, a widely adopted wireless protocol. In contrast to the passive day-long gastric residence achieved with prior ingestible electronics, advancement in multimaterial prototyping enables the GRE to reside in the hostile gastric environment for a maximum of 36 d and maintain ≈15 d of wireless electronics communications as evidenced by the studies in a porcine model. Indeed, the synergistic integration of reconfigurable gastric-residence structure, drug release modules, and wireless electronics could ultimately enable the next-generation remote diagnostic and automated therapeutic strategies.

An Enhanced Mechanistic Model For Capnography, With Application To CHF-COPD Discrimination.

Capnography records CO2 partial pressure in exhaled breath as a function of time or exhaled volume. Time-based capnography, which is our focus, is a point-of-care, noninvasive, effort-independent and widely available clinical monitoring modality. The generated waveform, or capnogram, reflects the ventilation-perfusion dynamics of the lung, and thus has value in the diagnosis of respiratory conditions such as chronic obstructive pulmonary disease (COPD). Effective discrimination between normal respiration and obstructive lung disease can be performed using capnogram-derived estimates of respiratory parameters in a simple mechanistic model of CO2 exhalation. We propose an enhanced mechanistic model that can capture specific capnogram characteristics in congestive heart failure (CHF) by incorporating a representation of the inertance associated with fluid in the lungs. The 4 associated parameters are estimated on a breath-by-breath basis by fitting the model output to the exhalations in the measured capnogram. Estimated parameters from 40 exhalations of 7 CHF and 7 COPD patients were used as a training set to design a quadratic discriminator in the parameter space, aimed at distinguishing between CHF and COPD patients. The area under the ROC curve for the training set was 0.94, and the corresponding equal-error-rate value of approximately 0.1 suggests classification accuracies of the order of 90% are attainable. Applying this discriminator without modification to 40 exhalations from each CHF and COPD patient in a fresh test set, and deciding on a simple majority basis whether the patient has CHF or COPD, results in correctly labeling all 8 out of the 8 CHF patients and 6 out of the 8 COPD patients in the test set, corresponding to a classification accuracy of 87.5%.

Exploring patterns enriched in a dataset with contrastive principal component analysis.

Visualization and exploration of high-dimensional data is a ubiquitous challenge across disciplines. Widely used techniques such as principal component analysis (PCA) aim to identify dominant trends in one dataset. However, in many settings we have datasets collected under different conditions, e.g., a treatment and a control experiment, and we are interested in visualizing and exploring patterns that are specific to one dataset. This paper proposes a method, contrastive principal component analysis (cPCA), which identifies low-dimensional structures that are enriched in a dataset relative to comparison data. In a wide variety of experiments, we demonstrate that cPCA with a background dataset enables us to visualize dataset-specific patterns missed by PCA and other standard methods. We further provide a geometric interpretation of cPCA and strong mathematical guarantees. An implementation of cPCA is publicly available, and can be used for exploratory data analysis in many applications where PCA is currently used.

Model-Based Estimation of Respiratory Parameters from Capnography, With Application to Diagnosing Obstructive Lung Disease.

OBJECTIVE: We use a single-alveolar-compartment model to describe the partial pressure of carbon dioxide in exhaled breath, as recorded in time-based capnography. Respiratory parameters are estimated using this model, and then related to the clinical status of patients with obstructive lung disease. METHODS: Given appropriate assumptions, we derive an analytical solution of the model, describing the exhalation segment of the capnogram. This solution is parametrized by alveolar CO2 concentration, dead-space fraction, and the time constant associated with exhalation. These quantities are estimated from individual capnogram data on a breath-by-breath basis. The model is applied to analyzing datasets from normal (n = 24) and chronic obstructive pulmonary disease (COPD) (n = 22) subjects, as well as from patients undergoing methacholine challenge testing for asthma (n = 22). RESULTS: A classifier based on linear discriminant analysis in logarithmic coordinates, using estimated dead-space fraction and exhalation time constant as features, and trained on data from five normal and five COPD subjects, yielded an area under the receiver operating characteristic curve (AUC) of 0.99 in classifying the remaining 36 subjects as normal or COPD. Bootstrapping with 50 replicas yielded a 95% confidence interval of AUCs from 0.96 to 1.00. For patients undergoing methacholine challenge testing, qualitatively meaningful trends were observed in the parameter variations over the course of the test. SIGNIFICANCE: A simple mechanistic model allows estimation of underlying respiratory parameters from the capnogram, and may be applied to diagnosis and monitoring of chronic and reversible obstructive lung disease.

Wireless Power Transfer to Millimeter-Sized Gastrointestinal Electronics Validated in a Swine Model.

Electronic devices placed in the gastrointestinal (GI) tract for prolonged periods have the potential to transform clinical evaluation and treatment. One challenge to the deployment of such gastroresident electronics is the difficulty in powering millimeter-sized electronics devices without using batteries, which compromise biocompatibility and long-term residence. We examined the feasibility of leveraging mid-field wireless powering to transfer power from outside of the body to electronics at various locations along the GI tract. Using simulations and ex vivo measurements, we designed mid-field antennas capable of operating efficiently in tissue at 1.2 GHz. These antennas were then characterized in vivo in five anesthetized pigs, by placing one antenna outside the body, and the other antenna inside the body endoscopically, at the esophagus, stomach, and colon. Across the animals tested, mean transmission efficiencies of -41.2, -36.1, and -34.6 dB were achieved in vivo while coupling power from outside the body to the esophagus, stomach, and colon, respectively. This corresponds to power levels of 37.5 µW, 123 µW and 173 µW received by antennas in the respective locations, while keeping radiation exposure levels below safety thresholds. These power levels are sufficient to wirelessly power a range of medical devices from outside of the body.

Model-based estimation of pulmonary compliance and resistance parameters from time-based capnography.

We propose a highly-simplified single-alveolus mechanistic model of lung mechanics and gas mixing that leads to an analytical solution for carbon dioxide partial pressure in exhaled breath, as measured by time-based capnography. Using this solution, we estimate physiological parameters of the lungs on a continuous, breath-by-breath basis. We validate our model with capnograms from 15 subjects responding positively (>20% FEV1 drop from baseline) to methacholine challenge, and subsequently recovering with bronchodilator treatment. Our results suggest that parameter estimates from capnography may provide discriminatory value for lung function comparable to spirometry, thus warranting more detailed study.