CP-Net: Instance-aware part segmentation network for biological cell parsing.

Instance segmentation of biological cells is important in medical image analysis for identifying and segmenting individual cells, and quantitative measurement of subcellular structures requires further cell-level subcellular part segmentation. Subcellular structure measurements are critical for cell phenotyping and quality analysis. For these purposes, instance-aware part segmentation network is first introduced to distinguish individual cells and segment subcellular structures for each detected cell. This approach is demonstrated on human sperm cells since the World Health Organization has established quantitative standards for sperm quality assessment. Specifically, a novel Cell Parsing Net (CP-Net) is proposed for accurate instance-level cell parsing. An attention-based feature fusion module is designed to alleviate contour misalignments for cells with an irregular shape by using instance masks as spatial cues instead of as strict constraints to differentiate various instances. A coarse-to-fine segmentation module is developed to effectively segment tiny subcellular structures within a cell through hierarchical segmentation from whole to part instead of directly segmenting each cell part. Moreover, a sperm parsing dataset is built including 320 annotated sperm images with five semantic subcellular part labels. Extensive experiments on the collected dataset demonstrate that the proposed CP-Net outperforms state-of-the-art instance-aware part segmentation networks.

Modeling and Compensation of Positioning Error in Micromanipulation.

In order to improve the positioning accuracy of the micromanipulation system, a comprehensive error model is first established to take into account the microscope nonlinear imaging distortion, camera installation error, and the mechanical displacement error of the motorized stage. A novel error compensation method is then proposed with distortion compensation coefficients obtained by the Levenberg-Marquardt optimization algorithm combined with the deduced nonlinear imaging model. The compensation coefficients for camera installation error and mechanical displacement error are derived from the rigid-body translation technique and image stitching algorithm. To validate the error compensation model, single shot and cumulative error tests were designed. The experimental results show that after the error compensation, the displacement errors were controlled within 0.25 µm when moving in a single direction and within 0.02 µm per 1000 µm when moving in multiple directions.

3D Morphology Measurement for Blastocyst Evaluation From "All Angles".

Measuring the 3D morphology of spherical cell aggregates is required in both biology and medicine. Traditional methods either use fluorescent labeling, which cause cell toxicity and are unsuitable for clinical treatment, or use 2D images to roughly estimate 3D morphology. To overcome these limitations, this paper presents a quantitative label-free 3D morphology measurement technique using multi-view images. This technique, for the first time, enables the morphological evaluation of a blastocyst (Day 5 embryo) from "all angles" for IVF treatment. In this technique, a spherical rotation scale invariant feature transform (SR-SIFT) is proposed to address feature distortions for the rotation matrix calculation of the multi-view images. U-Net with generalized Dice loss is used to segment individual trophectoderm (TE) cells and the inner cell mass (ICM) of the blastocyst. Based on the rotation matrices and the segmentation results, the 3D morphological parameters of the entire blastocyst were quantified. Experimental results showed that the error of rotation angle was less than 1 °, the Dice was 95.6% for TE segmentation and 92.3% for ICM segmentation, and the overall measurement error of clinically defined blastocyst parameters was less than 6.7%.

Development and evaluation of a live birth prediction model for evaluating human blastocysts from a retrospective study.

Background: In infertility treatment, blastocyst morphological grading is commonly used in clinical practice for blastocyst evaluation and selection, but has shown limited predictive power on live birth outcomes of blastocysts. To improve live birth prediction, a number of artificial intelligence (AI) models have been established. Most existing AI models for blastocyst evaluation only used images for live birth prediction, and the area under the receiver operating characteristic (ROC) curve (AUC) achieved by these models has plateaued at ~0.65. Methods: This study proposed a multimodal blastocyst evaluation method using both blastocyst images and patient couple's clinical features (e.g., maternal age, hormone profiles, endometrium thickness, and semen quality) to predict live birth outcomes of human blastocysts. To utilize the multimodal data, we developed a new AI model consisting of a convolutional neural network (CNN) to process blastocyst images and a multilayer perceptron to process patient couple's clinical features. The data set used in this study consists of 17,580 blastocysts with known live birth outcomes, blastocyst images, and patient couple's clinical features. Results: This study achieved an AUC of 0.77 for live birth prediction, which significantly outperforms related works in the literature. Sixteen out of 103 clinical features were identified to be predictors of live birth outcomes and helped improve live birth prediction. Among these features, maternal age, the day of blastocyst transfer, antral follicle count, retrieved oocyte number, and endometrium thickness measured before transfer are the top five features contributing to live birth prediction. Heatmaps showed that the CNN in the AI model mainly focuses on image regions of inner cell mass and trophectoderm (TE) for live birth prediction, and the contribution of TE-related features was greater in the CNN trained with the inclusion of patient couple's clinical features compared with the CNN trained with blastocyst images alone. Conclusions: The results suggest that the inclusion of patient couple's clinical features along with blastocyst images increases live birth prediction accuracy. Funding: Natural Sciences and Engineering Research Council of Canada and the Canada Research Chairs Program.

More than 50 million couples worldwide experience infertility. The most common treatment is in vitro fertilization (IVF). Fertility specialists collect eggs and sperm from the prospective parents. They combine the egg and sperm in a laboratory and allow the fertilized eggs to develop for five days into a multi-celled blastocyst. Then, the specialists select the healthiest blastocysts and return them to the patient's uterus. Since 1978, more than 8 million children have been conceived through IVF. Yet, only about 30% of IVF attempts result in a successful birth. As a result, fertility patients often undergo multiple rounds of IVF, which can be expensive and emotionally draining. Several factors determine IVF success, one of which is the health of the blastocysts selected for transfer to the uterus. Specialists select the blastocysts using several criteria. But these human assessments are subjective and inconsistent in predicting which ones are most likely to result in a successful birth. Recent studies suggest artificial intelligence technology may help select blastocysts. Liu et al. show that using artificial intelligence to assess blastocysts and fertility patient characteristics leads to more accurate predictions about which blastocysts are likely to result in a successful birth. In the experiments, the researchers trained an artificial intelligence computer program using pictures of 17,580 blastocysts with known birth outcomes and the parents' clinical characteristics. The model identified 16 parental factors associated with birth outcomes. The top 5 most predictive parental factors were maternal age, the day of blastocyst transfer to the uterus, how many eggs were present in the ovaries, the number of eggs retrieved and the thickness of the uterus lining. The program achieved the highest prediction of healthy births so far, compared to success rates listed in other studies. Artificial intelligence-aided blastocyte selection using patient and blastocyst characteristics may improve IVF success rates and reduce the number of treatment cycles patient couples undergo. Before specialists can use artificial intelligence in their clinics, they must conduct confirmatory clinical studies that enroll patient couples to compare conventional methods and artificial intelligence.

Staining-free, Automated Sperm Analysis for In Vitro Fertilization Lab Use.

PURPOSE: Computer-aided sperm analysis is typically used in andrology labs, not in in vitro fertilization labs, which requires staining for sperm morphology measurement. In in vitro fertilization labs, sperm analysis still relies on manual observation and suffers from subjectivity and inconsistency. We developed a system for automated measurement of sperm concentration, motility, and morphology without the need for sperm staining. The reproducibility and reliability of the system were evaluated. MATERIALS AND METHODS: Thirty-five fresh semen and 25 washed samples were obtained from male partners attending for fertility investigations. Sperm concentration, motility, and morphology were automatically measured simultaneously, leveraging robust sperm tracking for concentration and motility measurement and low contrast image segmentation for morphology measurement of live sperm. Reproducibility of sperm measurements was evaluated by intraclass correlation coefficients. Reliability of sperm measurement was evaluated by Passing and Bablok regression analysis and Bland-Altman analysis. RESULTS: Automated measurement of concentration, motility, and morphology had intraclass correlation coefficients higher than 0.97. The regression and Bland-Altman analysis indicated that automated measurement and off-line manual benchmarking with zoomed-in images were interchangeable. Further analysis on semen and washed samples and the measurement on progressive and nonprogressive motility also showed high reproducibility and reliability. CONCLUSIONS: Automated sperm analysis revealed high reproducibility and reliability. The system is designed for routine use in in vitro fertilization labs to perform quantitative sperm analysis on live samples.

Automated Denudation of Oocytes.

Denudation is a technique for removal of the cumulus cell mass from oocytes in clinical intracytoplasmic sperm injection (ICSI). Manual oocyte denudation requires long training hours and stringent skills, but still suffers from low yield rate and denudation efficiency due to human fatigue and skill variations across operators. To address these limitations, this paper reports a robotic system for automated oocyte denudation. In this system, several key techniques are proposed, including a vision-based contact detection method for measuring the relative z position between the micropipette tip and the dish substrate, recognition of oocytes and the surrounding cumulus cells, automated calibration algorithm for eliminating the misalignment angle, and automated control of the flow rate based on the model of oocyte dynamics during micropipette aspiration and deposition. Experiments on mouse oocytes demonstrated that the robotic denudation system achieved a high yield rate of 97.0 ± 2.8% and denudation efficiency of 95.0 ± 0.8%. Additionally, oocytes denuded by the robotic system showed comparable fertilization rate and developmental competence compared with manual denudation. Our robotic denudation system represents one step towards the automation and standardization of ICSI procedures.

Quantitative selection of single human sperm with high DNA integrity for intracytoplasmic sperm injection.

OBJECTIVE: To study at the single-cell level whether a sperm's motility and morphology parameters reflect its DNA integrity, and to establish a set of quantitative criteria for selecting single sperm with high DNA integrity. DESIGN: Prospective study. SETTING: In vitro fertilization center and university laboratories. PATIENT(S): Male patients undergoing infertility treatments. INTERVENTION(S): None. MAIN OUTCOME MEASURE(S): The motility and morphology parameters of each sperm were measured with the use of computer vision algorithms. The sperm was then aspirated and transferred for DNA fragmentation measurement by single-cell gel electrophoresis (comet assay). RESULT(S): We adapted the World Health Organization criteria, which were originally defined for semen analysis, and established a set of quantitative criteria for single-sperm selection in intracytoplasmic sperm injection. Sperm satisfying the criteria had significantly lower DNA fragmentation levels than the sample population. Both normal motility and normal morphology were required for a sperm to have low DNA fragmentation. The quantitative criteria were integrated into a software program for sperm selection. In blind tests in which our software and three embryologists selected sperm from the same patient samples, our software outperformed the embryologists and selected sperm with the highest DNA integrity. CONCLUSION(S): At the single-cell level, a sperm's motility and morphology parameters reflect its DNA integrity. The developed technique and criteria hold the potential to mitigate the risk factor of sperm DNA fragmentation in intracytoplasmic sperm injection.

Advances in sperm analysis: techniques, discoveries and applications.

Infertility affects one in six couples worldwide, and fertility continues to deteriorate globally, partly owing to a decline in semen quality. Sperm analysis has a central role in diagnosing and treating male factor infertility. Many emerging techniques, such as digital holography, super-resolution microscopy and next-generation sequencing, have been developed that enable improved analysis of sperm motility, morphology and genetics to help overcome limitations in accuracy and consistency, and improve sperm selection for infertility treatment. These techniques have also improved our understanding of fundamental sperm physiology by enabling discoveries in sperm behaviour and molecular structures. Further progress in sperm analysis and integrating these techniques into laboratories and clinics requires multidisciplinary collaboration, which will increase discovery and improve clinical outcomes.

Automated motility and morphology measurement of live spermatozoa.

BACKGROUND: Automated sperm analysis has wide applications in infertility diagnosis. Existing systems are not able to measure sperm count and both motility and morphology of individual live spermatozoa. Morphology measurement requires invasive staining, making the spermatozoa after morphology measurement not applicable to infertility treatment. OBJECTIVE: To evaluate the reproducibility and reliability of automated measurement of individual live sperm's motility and morphology. MATERIALS AND METHODS: Fresh semen samples were obtained from twenty male partners attending for fertility investigations. The system firstly measured motility for all the spermatozoa within the field of view under a low magnification (20×), then a spermatozoa of interest is selected by the user and automatically relocated by the system after switching to a high magnification (100×) for morphology measurement. Reproducibility of sperm measurements was evaluated by intraclass correlation coefficients on consecutive measurement. Reliability of motility and morphology measurement was evaluated by tracking error rate and limits of agreement, respectively, with manual measurement as benchmark. RESULTS: Measurement of all motility and morphology parameters had intraclass correlation coefficients higher than 0.94. Sperm motility measurement had a tracking error rate of 2.1%. Limit of agreement analysis indicated that automated measurement and manual measurement of sperm morphology were interchangeable. Automated measurement of all morphology parameters was not statistically different from manual measurement, as confirmed by the paired sample t test. DISCUSSION: Automated motility and morphology measurement of single sperm revealed high reproducibility and reliability. The system also achieved a high efficiency for motility and morphology measurement. In addition to the intracytoplasmic sperm injection (ICSI) samples with polyvinylpyrrolidone (PVP), the developed sperm measurement technique is also effective for analyzing semen and washed samples. The system provides a valuable tool for quantitative measurement and selection of single spermatozoa for ICSI. It can also be used for sperm motility and morphology analysis in andrology laboratories.

Robotic Immobilization of Motile Sperm for Clinical Intracytoplasmic Sperm Injection.

OBJECTIVE: In clinical intracytoplasmic sperm injection (ICSI), a motile sperm must be immobilized before insertion into an oocyte. This paper aims to develop a robotic system for automated tracking, orientation control, and immobilization of motile sperms for clinical ICSI applications. METHODS: We adapt the probabilistic data association filter by adding sperm head orientation into state variables for robustly tracking the sperm head and estimating sperm tail positions under interfering conditions. The robotic system also utilizes a motorized rotational microscopy stage and a new visual servo control strategy that predicts and compensates for sperm movements to actively adjust sperm orientation for immobilizing a sperm swimming in any direction. RESULTS: The system robustly tracked sperm head with a tracking success rate of 96.0% and estimated sperm tail position with an accuracy of 1.08 µm under clinical conditions where the occlusion of the target sperm and interference from other sperms occur. Experimental results from robotic immobilization of 400 sperms confirmed that the system achieved a consistent immobilization success rate of 94.5%, independent of sperm velocity or swimming direction. CONCLUSION: Our adapted tracking algorithm effectively distinguishes the target sperm from interfering sperms. Predicting and compensating for sperm movements significantly reduce the positioning error during sperm orientation control. These features make the robotic system suitable for automated sperm immobilization. SIGNIFICANCE: The robotic system eliminates stringent skill requirements in manual sperm immobilization. It is capable of manipulating sperms swimming in an arbitrary direction with a high success rate.

Automated Non-Invasive Measurement of Single Sperm's Motility and Morphology.

Measuring cell motility and morphology is important for revealing their functional characteristics. This paper presents automation techniques that enable automated, non-invasive measurement of motility and morphology parameters of single sperm. Compared to the status quo of qualitative estimation of single sperm's motility and morphology manually, the automation techniques provide quantitative data for embryologists to select a single sperm for intracytoplasmic sperm injection. An adapted joint probabilistic data association filter was used for multi-sperm tracking and tackled challenges of identifying sperms that intersect or have small spatial distances. Since the standard differential interference contrast (DIC) imaging method has side illumination effect which causes inherent inhomogeneous image intensity and poses difficulties for accurate sperm morphology measurement, we integrated total variation norm into the quadratic cost function method, which together effectively removed inhomogeneous image intensity and retained sperm's subcellular structures after DIC image reconstruction. In order to relocate the same sperm of interest identified under low magnification after switching to high magnification, coordinate transformation was conducted to handle the changes in the field of view caused by magnification switch. The sperm's position after magnification switch was accurately predicted by accounting for the sperm's swimming motion during magnification switch. Experimental results demonstrated an accuracy of 95.6% in sperm motility measurement and an error <10% in morphology measurement.

A Flexure-Guided Piezo Drill for Penetrating the Zona Pellucida of Mammalian Oocytes.

Mammalian oocytes such as mouse oocytes have a highly elastic outer membrane, zona pellucida (ZP) that cannot be penetrated without significantly deforming the oocyte, even with a sharp micropipette. Piezo drill devices leverage lateral and axial vibration of the micropipette to accomplish ZP penetration with greatly reduced oocyte deformation. However, existing piezo drills all rely on a large lateral micropipette vibration amplitude ( 20 ) and a small axial vibration amplitude (0.1 ). The very large lateral vibration amplitude has been deemed to be necessary for ZP penetration although it also induces larger oocyte deformation and more oocyte damage. This paper reports on a new piezo drill device that uses a flexure guidance mechanism and a systematically designed pulse train with an appropriate base frequency. Both simulation and experimental results demonstrate that a small lateral vibration amplitude (e.g., 2 ) and an axial vibration amplitude as large as 1.2 were achieved. Besides achieving 100% effectiveness in the penetration of mouse oocytes (n = 45), the new piezo device during ZP penetration induced a small oocyte deformation of 3.4 versus larger than 10 using existing piezo drill devices.