The use of voting ensembles to improve the accuracy of deep neural networks as a non-invasive method to predict embryo ploidy status.

PURPOSE: To determine if creating voting ensembles combining convolutional neural networks (CNN), support vector machine (SVM), and multi-layer neural networks (NN) alongside clinical parameters improves the accuracy of artificial intelligence (AI) as a non-invasive method for predicting aneuploidy. METHODS: A cohort of 699 day 5 PGT-A tested blastocysts was used to train, validate, and test a CNN to classify embryos as euploid/aneuploid. All embryos were analyzed using a modified FAST-SeqS next-generation sequencing method. Patient characteristics such as maternal age, AMH level, paternal sperm quality, and total number of normally fertilized (2PN) embryos were processed using SVM and NN. To improve model performance, we created voting ensembles using CNN, SVM, and NN to combine our imaging data with clinical parameter variations. Statistical significance was evaluated with a one-sample t-test with 2 degrees of freedom. RESULTS: When assessing blastocyst images alone, the CNN test accuracy was 61.2% (± 1.32% SEM, n = 3 models) in correctly classifying euploid/aneuploid embryos (n = 140 embryos). When the best CNN model was assessed as a voting ensemble, the test accuracy improved to 65.0% (AMH; p = 0.1), 66.4% (maternal age; p = 0.06), 65.7% (maternal age, AMH; p = 0.08), 66.4% (maternal age, AMH, number of 2PNs; p = 0.06), and 71.4% (maternal age, AMH, number of 2PNs, sperm quality; p = 0.02) (n = 140 embryos). CONCLUSIONS: By combining CNNs with patient characteristics, voting ensembles can be created to improve the accuracy of classifying embryos as euploid/aneuploid from CNN alone, allowing for AI to serve as a potential non-invasive method to aid in karyotype screening and selection of embryos.

Advancements in the future of automating micromanipulation techniques in the IVF laboratory using deep convolutional neural networks.

PURPOSE: To determine if deep learning artificial intelligence algorithms can be used to accurately identify key morphologic landmarks on oocytes and cleavage stage embryo images for micromanipulation procedures such as intracytoplasmic sperm injection (ICSI) or assisted hatching (AH). METHODS: Two convolutional neural network (CNN) models were trained, validated, and tested over three replicates to identify key morphologic landmarks used to guide embryologists when performing micromanipulation procedures. The first model (CNN-ICSI) was trained (n = 13,992), validated (n = 1920), and tested (n = 3900) to identify the optimal location for ICSI through polar body identification. The second model (CNN-AH) was trained (n = 13,908), validated (n = 1908), and tested (n = 3888) to identify the optimal location for AH on the zona pellucida that maximizes distance from healthy blastomeres. RESULTS: The CNN-ICSI model accurately identified the polar body and corresponding optimal ICSI location with 98.9% accuracy (95% CI 98.5-99.2%) with a receiver operator characteristic (ROC) with micro and macro area under the curves (AUC) of 1. The CNN-AH model accurately identified the optimal AH location with 99.41% accuracy (95% CI 99.11-99.62%) with a ROC with micro and macro AUCs of 1. CONCLUSION: Deep CNN models demonstrate powerful potential in accurately identifying key landmarks on oocytes and cleavage stage embryos for micromanipulation. These findings are novel, essential stepping stones in the automation of micromanipulation procedures.

Validation of a smartphone-based device to measure concentration, motility, and morphology in swine ejaculates.

Assessment of swine semen quality is important as it is used as an estimate of the fertility of an ejaculate. There are many methods to measure sperm morphology, concentration, and motility, however, some methods require expensive instrumentation or are not easy to use on-farm. A portable, low-cost, automated device could provide the potential to assess semen quality in field conditions. The objective of this study was to validate the use of Fertile-Eyez (FE), a smartphone-based device, to measure sperm concentration, total motility, and morphology in boar ejaculates. Semen from six sexually mature boars were collected and mixed to create a total of 18 unique semen samples for system evaluations. Each sample was then diluted to 1:4, 1:8, 1:10, and 1:16 (for concentration only) with Androhep Plus semen extender (n = 82 total). Sperm concentration was evaluated using FE and compared to results measured using a Nucleocounter and computer assisted sperm analysis (CASA: Ceros II, Hamilton Thorne). Sperm motility was evaluated using FE and CASA. Sperm morphological assessments were evaluated by a single technician manually counting abnormalities and compared to FE deep-learning technology. Data were analyzed using both descriptive statistics (mean, standard deviation, intra-assay coefficient of variance, and residual standard deviation [RSD]) and statistical tests (correlation analysis between devices and Bland-Altman methods). Concentration analysis was strongly correlated (n = 18; r > 0.967; P < 0.0001) among all devices and dilutions. Analysis of motility showed moderate correlation and was significant when all dilutions are analyzed together (n = 54; r = 0.558; P < 0.001). The regression analysis for motility also showed the RSD as 3.95% between FE and CASA indicating a tight fit between devices. This RSD indicates that FE can find boars with unacceptable motility (boars for example with less than 70%) which impact fertility and litter size. The Bland-Altman analysis showed that FE-estimated morphological assessment and the conventionally estimated morphological score were similar, with a mean difference of ~1% (%95 Limits of Agreement: -6.2 to 8.1; n = 17). The results of this experiment demonstrate that FE, a portable and automated smartphone-based device, is capable of assessing concentration, motility, and morphology of boar semen samples.

Using artificial intelligence to avoid human error in identifying embryos: a retrospective cohort study.

PURPOSE: To determine whether convolutional neural networks (CNN) can be used to accurately ascertain the patient identity (ID) of cleavage and blastocyst stage embryos based on image data alone. METHODS: A CNN model was trained and validated over three replicates on a retrospective cohort of 4889 time-lapse embryo images. The algorithm processed embryo images for each patient and produced a unique identification key that was associated with the patient ID at a timepoint on day 3 (~ 65 hours post-insemination (hpi)) and day 5 (~ 105 hpi) forming our data library. When the algorithm evaluated embryos at a later timepoint on day 3 (~ 70 hpi) and day 5 (~ 110 hpi), it generates another key that was matched with the patient's unique key available in the library. This approach was tested using 400 patient embryo cohorts on day 3 and day 5 and number of correct embryo identifications with the CNN algorithm was measured. RESULTS: CNN technology matched the patient identification within random pools of 8 patient embryo cohorts on day 3 with 100% accuracy (n = 400 patients; 3 replicates). For day 5 embryo cohorts, the accuracy within random pools of 8 patients was 100% (n = 400 patients; 3 replicates). CONCLUSIONS: This study describes an artificial intelligence-based approach for embryo identification. This technology offers a robust witnessing step based on unique morphological features of each embryo. This technology can be integrated with existing imaging systems and laboratory protocols to improve specimen tracking.

SARS-CoV-2 RNA Detection by a Cellphone-Based Amplification-Free System with CRISPR/CAS-Dependent Enzymatic (CASCADE) Assay.

CRISPR (Clustered regularly interspaced short palindromic repeats)-based diagnostic technologies have emerged as a promising alternative to accelerate delivery of SARS-CoV-2 molecular detection at the point of need. However, efficient translation of CRISPR-diagnostic technologies to field application is still hampered by dependence on target amplification and by reliance on fluorescence-based results readout. Herein, an amplification-free CRISPR/Cas12a-based diagnostic technology for SARS-CoV-2 RNA detection is presented using a smartphone camera for results readout. This method, termed Cellphone-based amplification-free system with CRISPR/CAS-dependent enzymatic (CASCADE) assay, relies on mobile phone imaging of a catalase-generated gas bubble signal within a microfluidic channel and does not require any external hardware optical attachments. Upon specific detection of a SARS-CoV-2 reverse-transcribed DNA/RNA heteroduplex target (orf1ab) by the ribonucleoprotein complex, the transcleavage collateral activity of the Cas12a protein on a Catalase:ssDNA probe triggers the bubble signal on the system. High analytical sensitivity in signal detection without previous target amplification (down to 50 copies µL-1) is observed in spiked samples, in ≈71 min from sample input to results readout. With the aid of a smartphone vision tool, high accuracy (AUC = 1.0; CI: 0.715 - 1.00) is achieved when the CASCADE system is tested with nasopharyngeal swab samples of PCR-positive COVID-19 patients.

Adaptive adversarial neural networks for the analysis of lossy and domain-shifted datasets of medical images.

In machine learning for image-based medical diagnostics, supervised convolutional neural networks are typically trained with large and expertly annotated datasets obtained using high-resolution imaging systems. Moreover, the network's performance can degrade substantially when applied to a dataset with a different distribution. Here, we show that adversarial learning can be used to develop high-performing networks trained on unannotated medical images of varying image quality. Specifically, we used low-quality images acquired using inexpensive portable optical systems to train networks for the evaluation of human embryos, the quantification of human sperm morphology and the diagnosis of malarial infections in the blood, and show that the networks performed well across different data distributions. We also show that adversarial learning can be used with unlabelled data from unseen domain-shifted datasets to adapt pretrained supervised networks to new distributions, even when data from the original distribution are not available. Adaptive adversarial networks may expand the use of validated neural-network models for the evaluation of data collected from multiple imaging systems of varying quality without compromising the knowledge stored in the network.

Deep learning early warning system for embryo culture conditions and embryologist performance in the ART laboratory.

Staff competency is a crucial component of the in vitro fertilization (IVF) laboratory quality management system because it impacts clinical outcomes and informs the key performance indicators (KPIs) used to continuously monitor and assess culture conditions. Contemporary quality control and assurance in the IVF lab can be automated (collect, store, retrieve, and analyze), to elevate quality control and assurance beyond the cursory monthly review. Here we demonstrate that statistical KPI monitoring systems for individual embryologist performance and culture conditions can be detected by artificial intelligence systems to provide systemic, early detection of adverse outcomes, and identify clinically relevant shifts in pregnancy rates, providing critical validation for two statistical process controls proposed in the Vienna Consensus Document; intracytoplasmic sperm injection (ICSI) fertilization rate and day 3 embryo quality.

Evaluation of deep convolutional neural networks in classifying human embryo images based on their morphological quality.

A critical factor that influences the success of an in-vitro fertilization (IVF) treatment cycle is the quality of the transferred embryo. Embryo morphology assessments, conventionally performed through manual microscopic analysis suffer from disparities in practice, selection criteria, and subjectivity due to the experience of the embryologist. Convolutional neural networks (CNNs) are powerful, promising algorithms with significant potential for accurate classifications across many object categories. Network architectures and hyper-parameters affect the efficiency of CNNs for any given task. Here, we evaluate multi-layered CNNs developed from scratch and popular deep-learning architectures such as Inception v3, ResNET-50, Inception-ResNET-v2, NASNetLarge, ResNeXt-101, ResNeXt-50, and Xception in differentiating between embryos based on their morphological quality at 113 h post insemination (hpi). Xception performed the best in differentiating between the embryos based on their morphological quality.

Mobile Health (mHealth) Viral Diagnostics Enabled with Adaptive Adversarial Learning.

Deep-learning (DL)-based image processing has potential to revolutionize the use of smartphones in mobile health (mHealth) diagnostics of infectious diseases. However, the high variability in cellphone image data acquisition and the common need for large amounts of specialist-annotated images for traditional DL model training may preclude generalizability of smartphone-based diagnostics. Here, we employed adversarial neural networks with conditioning to develop an easily reconfigurable virus diagnostic platform that leverages a dataset of smartphone-taken microfluidic chip photos to rapidly generate image classifiers for different target pathogens on-demand. Adversarial learning was also used to augment this real image dataset by generating 16,000 realistic synthetic microchip images, through style generative adversarial networks (StyleGAN). We used this platform, termed smartphone-based pathogen detection resource multiplier using adversarial networks (SPyDERMAN), to accurately detect different intact viruses in clinical samples and to detect viral nucleic acids through integration with CRISPR diagnostics. We evaluated the performance of the system in detecting five different virus targets using 179 patient samples. The generalizability of the system was confirmed by rapid reconfiguration to detect SARS-CoV-2 antigens in nasal swab samples (n = 62) with 100% accuracy. Overall, the SPyDERMAN system may contribute to epidemic preparedness strategies by providing a platform for smartphone-based diagnostics that can be adapted to a given emerging viral agent within days of work.

Performance of a deep learning based neural network in the selection of human blastocysts for implantation.

Deep learning in in vitro fertilization is currently being evaluated in the development of assistive tools for the determination of transfer order and implantation potential using time-lapse data collected through expensive imaging hardware. Assistive tools and algorithms that can work with static images, however, can help in improving the access to care by enabling their use with images acquired from traditional microscopes that are available to virtually all fertility centers. Here, we evaluated the use of a deep convolutional neural network (CNN), trained using single timepoint images of embryos collected at 113 hr post-insemination, in embryo selection amongst 97 clinical patient cohorts (742 embryos) and observed an accuracy of 90% in choosing the highest quality embryo available. Furthermore, a CNN trained to assess an embryo's implantation potential directly using a set of 97 euploid embryos capable of implantation outperformed 15 trained embryologists (75.26% vs. 67.35%, p<0.0001) from five different fertility centers.

Around one in seven couples have trouble conceiving, which means there is a high demand for solutions such as in vitro fertilization, also known as IVF. This process involves fertilizing and developing embryos in the laboratory and then selecting a few to implant into the womb of the patient. IVF, however, only has a 30% success rate, is expensive and can be both mentally and physically taxing for patients. Selecting the right embryos to implant is therefore extremely important, as this increases the chance of success, minimizes complications and ensures the baby will be healthy. Currently the tools available for making this decision are limited, highly subjective, time-consuming, and often extremely expensive. As a result, embryologists often rely on their experience and observational skills when choosing which embryos to implant, which can lead to a lot of variability. An automated system based on artificial intelligence (AI) could therefore improve IVF success rates by assisting embryologists with this decision and ensuring more consistent results. The AI system could learn how embryos develop over time and then uses this information to select the best embryos to implant from just a single image. This would offer a cheaper alternative to current analysis tools that are only available at the most expensive IVF clinics. Now, Bormann, Kanakasabapathy, Thirumalaraj et al. have developed an AI system for IVF based on thousands of images of embryos. Using individual images, the system selected embryos of a comparable quality to those selected by a human specialist. It also showed a greater ability to identify embryos that will lead to successful implantation. Indeed, the software outperformed 15 embryologists from five different centers across the United States in detecting which embryos were most likely to implant out of a group of high-quality embryos with few visible differences. Artificial intelligence has many potential applications to support expert clinical decision-making. Systems like these could improve success, reduce errors and lead to faster, cheaper and more accessible results. Beyond immediate IVF applications, this system could also be used in research and industry to help understand differences in embryo quality.

Consistency and objectivity of automated embryo assessments using deep neural networks.

OBJECTIVE: To evaluate the consistency and objectivity of deep neural networks in embryo scoring and making disposition decisions for biopsy and cryopreservation in comparison to grading by highly trained embryologists. DESIGN: Prospective double-blind study using retrospective data. SETTING: U.S.-based large academic fertility center. PATIENTS: Not applicable. INTERVENTION(S): Embryo images (748 recorded at 70 hours postinsemination [hpi]) and 742 at 113 hpi) were used to evaluate embryologists and neural networks in embryo grading. The performance of 10 embryologists and a neural network were also evaluated in disposition decision making using 56 embryos. MAIN OUTCOME MEASURES: Coefficients of variation (%CV) and measures of consistencies were compared. RESULTS: Embryologists exhibited a high degree of variability (%CV averages: 82.84% for 70 hpi and 44.98% for 113 hpi) in grading embryo. When selecting blastocysts for biopsy or cryopreservation, embryologists had an average consistency of 52.14% and 57.68%, respectively. The neural network outperformed the embryologists in selecting blastocysts for biopsy and cryopreservation with a consistency of 83.92%. Cronbach's α analysis revealed an α coefficient of 0.60 for the embryologists and 1.00 for the network. CONCLUSIONS: The results of our study show a high degree of interembryologist and intraembryologist variability in scoring embryos, likely due to the subjective nature of traditional morphology grading. This may ultimately lead to less precise disposition decisions and discarding of viable embryos. The application of a deep neural network, as shown in our study, can introduce improved reliability and high consistency during the process of embryo selection and disposition, potentially improving outcomes in an embryology laboratory.

Development and evaluation of inexpensive automated deep learning-based imaging systems for embryology.

Embryo assessment and selection is a critical step in an in vitro fertilization (IVF) procedure. Current embryo assessment approaches such as manual microscopy analysis done by embryologists or semi-automated time-lapse imaging systems are highly subjective, time-consuming, or expensive. Availability of cost-effective and easy-to-use hardware and software for embryo image data acquisition and analysis can significantly empower embryologists towards more efficient clinical decisions both in resource-limited and resource-rich settings. Here, we report the development of two inexpensive (<$100 and <$5) and automated imaging platforms that utilize advances in artificial intelligence (AI) for rapid, reliable, and accurate evaluations of embryo morphological qualities. Using a layered learning approach, we have shown that network models pre-trained with high quality embryo image data can be re-trained using data recorded on such low-cost, portable optical systems for embryo assessment and classification when relatively low-resolution image data are used. Using two test sets of 272 and 319 embryo images recorded on the reported stand-alone and smartphone optical systems, we were able to classify embryos based on their cell morphology with >90% accuracy.

Automated smartphone-based system for measuring sperm viability, DNA fragmentation, and hyaluronic binding assay score.

The fundamental test for male infertility, semen analysis, is mostly a manually performed subjective and time-consuming process and the use of automated systems has been cost prohibitive. We have previously developed an inexpensive smartphone-based system for at-home male infertility screening through automatic and rapid measurement of sperm concentration and motility. Here, we assessed the feasibility of using a similar smartphone-based system for laboratory use in measuring: a) Hyaluronan Binding Assay (HBA) score, a quantitative score describing the sperm maturity and fertilization potential in a semen sample, b) sperm viability, which assesses sperm membrane integrity, and c) sperm DNA fragmentation that assesses the degree of DNA damage. There was good correlation between the manual analysis and smartphone-based analysis for the HBA score when the device was tested with 31 fresh, unprocessed human semen samples. The smartphone-based approach performed with an accuracy of 87% in sperm classification when the HBA score was set at manufacturer's threshold of 80. Similarly, the sperm viability and DNA fragmentation tests were also shown to be compatible with the smartphone-based system when tested with 102 and 47 human semen samples, respectively.

An inexpensive smartphone-based device for point-of-care ovulation testing.

The ability to accurately predict ovulation at-home using low-cost point-of-care diagnostics can be of significant help for couples who prefer natural family planning. Detecting ovulation-specific hormones in urine samples and monitoring basal body temperature are the current commonly home-based methods used for ovulation detection; however, these methods, relatively, are expensive for prolonged use and the results are difficult to comprehend. Here, we report a smartphone-based point-of-care device for automated ovulation testing using artificial intelligence (AI) by detecting fern patterns in a small volume (<100 µL) of saliva that is air-dried on a microfluidic device. We evaluated the performance of the device using artificial saliva and human saliva samples and observed that the device showed >99% accuracy in effectively predicting ovulation.