BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets.

BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in many neuroscience laboratories interested in neuron tracing. Here, we report generated gold standard manual annotations for a subset of the available imaging datasets and quantified tracing quality for 35 automatic tracing algorithms. The goal of generating such a hand-curated diverse dataset is to advance the development of tracing algorithms and enable generalizable benchmarking. Together with image quality features, we pooled the data in an interactive web application that enables users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and tracing data, and benchmarking of automatic tracing algorithms in user-defined data subsets. The image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. We observed that diverse algorithms can provide complementary information to obtain accurate results and developed a method to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms in noisy datasets. However, specific algorithms may outperform the consensus tree strategy in specific imaging conditions. Finally, to aid users in predicting the most accurate automatic tracing results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic tracings.

CW-NET for multitype cell detection and classification in bone marrow examination and mitotic figure examination.

MOTIVATION: Bone marrow (BM) examination is one of the most important indicators in diagnosing hematologic disorders and is typically performed under the microscope via oil-immersion objective lens with a total 100× objective magnification. On the other hand, mitotic detection and identification is critical not only for accurate cancer diagnosis and grading but also for predicting therapy success and survival. Fully automated BM examination and mitotic figure examination from whole-slide images is highly demanded but challenging and poorly explored. First, the complexity and poor reproducibility of microscopic image examination are due to the cell type diversity, delicate intralineage discrepancy within the multitype cell maturation process, cells overlapping, lipid interference and stain variation. Second, manual annotation on whole-slide images is tedious, laborious and subject to intraobserver variability, which causes the supervised information restricted to limited, easily identifiable and scattered cells annotated by humans. Third, when the training data are sparsely labeled, many unlabeled objects of interest are wrongly defined as background, which severely confuses AI learners. RESULTS: This article presents an efficient and fully automatic CW-Net approach to address the three issues mentioned above and demonstrates its superior performance on both BM examination and mitotic figure examination. The experimental results demonstrate the robustness and generalizability of the proposed CW-Net on a large BM WSI dataset with 16 456 annotated cells of 19 BM cell types and a large-scale WSI dataset for mitotic figure assessment with 262 481 annotated cells of five cell types. AVAILABILITY AND IMPLEMENTATION: An online web-based system of the proposed method has been created for demonstration (see https://youtu.be/MRMR25Mls1A).

Deep Learning Can Predict Bevacizumab Therapeutic Effect and Microsatellite Instability Directly from Histology in Epithelial Ovarian Cancer.

Epithelial ovarian cancer (EOC) remains a significant cause of mortality among gynecologic cancers, with the majority of cases being diagnosed at an advanced stage. Before targeted therapies were available, EOC treatment relied largely on debulking surgery and platinum-based chemotherapy. Vascular endothelial growth factors have been identified as inducing tumor angiogenesis. According to several clinical trials, anti-vascular endothelial growth factor-targeted therapy with bevacizumab was effective in all phases of EOC treatment. However, there are currently no biomarkers accessible for regular therapeutic use despite the importance of patient selection. Microsatellite instability (MSI), caused by a deficiency of the DNA mismatch repair system, is a molecular abnormality observed in EOC associated with Lynch syndrome. Recent evidence suggests that angiogenesis and MSI are interconnected. Developing predictive biomarkers, which enable the selection of patients who might benefit from bevacizumab-targeted therapy or immunotherapy, is critical for realizing personalized precision medicine. In this study, we developed 2 improved deep learning methods that eliminate the need for laborious detailed image-wise annotations by pathologists and compared them with 3 state-of-the-art methods to not only predict the efficacy of bevacizumab in patients with EOC using mismatch repair protein immunostained tissue microarrays but also predict MSI status directly from histopathologic images. In prediction of therapeutic outcomes, the 2 proposed methods achieved excellent performance by obtaining the highest mean sensitivity and specificity score using MSH2 or MSH6 markers and outperformed 3 state-of-the-art deep learning methods. Moreover, both statistical analysis results, using Cox proportional hazards model analysis and Kaplan-Meier progression-free survival analysis, confirm that the 2 proposed methods successfully differentiate patients with positive therapeutic effects and lower cancer recurrence rates from patients experiencing disease progression after treatment (P < .01). In prediction of MSI status directly from histopathology images, our proposed method also achieved a decent performance in terms of mean sensitivity and specificity score even for imbalanced data sets for both internal validation using tissue microarrays from the local hospital and external validation using whole section slides from The Cancer Genome Atlas archive.

Annotation-Free Deep Learning-Based Prediction of Thyroid Molecular Cancer Biomarker BRAF (V600E) from Cytological Slides.

Thyroid cancer is the most common endocrine cancer. Papillary thyroid cancer (PTC) is the most prevalent form of malignancy among all thyroid cancers arising from follicular cells. Fine needle aspiration cytology (FNAC) is a non-invasive method regarded as the most cost-effective and accurate diagnostic method of choice in diagnosing PTC. Identification of BRAF (V600E) mutation in thyroid neoplasia may be beneficial because it is specific for malignancy, implies a worse prognosis, and is the target for selective BRAF inhibitors. To the authors' best knowledge, this is the first automated precision oncology framework effectively predict BRAF (V600E) immunostaining result in thyroidectomy specimen directly from Papanicolaou-stained thyroid fine-needle aspiration cytology and ThinPrep cytological slides, which is helpful for novel targeted therapies and prognosis prediction. The proposed deep learning (DL) framework is evaluated on a dataset of 118 whole slide images. The results show that the proposed DL-based technique achieves an accuracy of 87%, a precision of 94%, a sensitivity of 91%, a specificity of 71% and a mean of sensitivity and specificity at 81% and outperformed three state-of-the-art deep learning approaches. This study demonstrates the feasibility of DL-based prediction of critical molecular features in cytological slides, which not only aid in accurate diagnosis but also provide useful information in guiding clinical decision-making in patients with thyroid cancer. With the accumulation of data and the continuous advancement of technology, the performance of DL systems is expected to be improved in the near future. Therefore, we expect that DL can provide a cost-effective and time-effective alternative tool for patients in the era of precision oncology.

Current Progress of Platelet-Rich Derivatives in Cartilage and Joint Repairs.

In recent years, several types of platelet concentrates have been investigated and applied in many fields, particularly in the musculoskeletal system. Platelet-rich fibrin (PRF) is an autologous biomaterial, a second-generation platelet concentrate containing platelets and growth factors in the form of fibrin membranes prepared from the blood of patients without additives. During tissue regeneration, platelet concentrates contain a higher percentage of leukocytes and a flexible fibrin net as a scaffold to improve cell migration in angiogenic, osteogenic, and antibacterial capacities during tissue regeneration. PRF enables the release of molecules over a longer period, which promotes tissue healing and regeneration. The potential of PRF to simulate the physiology and immunology of wound healing is also due to the high concentrations of released growth factors and anti-inflammatory cytokines that stimulate vessel formation, cell proliferation, and differentiation. These products have been used safely in clinical applications because of their autologous origin and minimally invasive nature. We focused on a narrative review of PRF therapy and its effects on musculoskeletal, oral, and maxillofacial surgeries and dermatology. We explored the components leading to the biological activity and the published preclinical and clinical research that supports its application in musculoskeletal therapy. The research generally supports the use of PRF as an adjuvant for various chronic muscle, cartilage, and tendon injuries. Further clinical trials are needed to prove the benefits of utilizing the potential of PRF.

Fully Automatic Registration Methods for Chest X-Ray Images.

PURPOSE: Image registration is important in medical applications accomplished by improving healthcare technology in recent years. Various studies have been proposed in medical applications, including clinical track of events and updating the treatment plan for radiotherapy and surgery. This study presents a fully automatic registration system for chest X-ray images to generate fusion results for difference analysis. Using the accurate alignment of the proposed system, the fusion result indicates the differences in the thoracic area during the treatment process. METHODS: The proposed method consists of a data normalization method, a hybrid L-SVM model to detect lungs, ribs and clavicles for object recognition, a landmark matching algorithm, two-stage transformation approaches and a fusion method for difference analysis to highlight the differences in the thoracic area. In evaluation, a preliminary test was performed to compare three transformation models, with a full evaluation process to compare the proposed method with two existing elastic registration methods. RESULTS: The results show that the proposed method produces significantly better results than two benchmark methods (P-value ≤ 0.001). The proposed system achieves the lowest mean registration error distance (MRED) (8.99 mm, 23.55 pixel) and the lowest mean registration error ratio (MRER) w.r.t. the length of image diagonal (1.61%) compared to the two benchmark approaches with MRED (15.64 mm, 40.97 pixel) and (180.5 mm, 472.69 pixel) and MRER (2.81%) and (32.51%), respectively. CONCLUSIONS: The experimental results show that the proposed method is capable of accurately aligning the chest X-ray images acquired at different times, assisting doctors to trace individual health status, evaluate treatment effectiveness and monitor patient recovery progress for thoracic diseases.

A benchmark for comparing precision medicine methods in thyroid cancer diagnosis using tissue microarrays.

Motivation: The aim of precision medicine is to harness new knowledge and technology to optimize the timing and targeting of interventions for maximal therapeutic benefit. This study explores the possibility of building AI models without precise pixel-level annotation in prediction of the tumor size, extrathyroidal extension, lymph node metastasis, cancer stage and BRAF mutation in thyroid cancer diagnosis, providing the patients' background information, histopathological and immunohistochemical tissue images. Results: A novel framework for objective evaluation of automatic patient diagnosis algorithms has been established under the auspices of the IEEE International Symposium on Biomedical Imaging 2017- A Grand Challenge for Tissue Microarray Analysis in Thyroid Cancer Diagnosis. Here, we present the datasets, methods and results of the challenge and lay down the principles for future uses of this benchmark. The main contributions of the challenge include the creation of the data repository of tissue microarrays; the creation of the clinical diagnosis classification data repository of thyroid cancer; and the definition of objective quantitative evaluation for comparison and ranking of the algorithms. With this benchmark, three automatic methods for predictions of the five clinical outcomes have been compared, and detailed quantitative evaluation results are presented in this paper. Based on the quantitative evaluation results, we believe automatic patient diagnosis is still a challenging and unsolved problem. Availability and implementation: The datasets and the evaluation software will be made available to the research community, further encouraging future developments in this field. (http://www-o.ntust.edu.tw/cvmi/ISBI2017/). Contact: cweiwang@mail.ntust.edu.tw. Supplementary information: Supplementary data are available at Bioinformatics online.

Improved image alignment method in application to X-ray images and biological images.

MOTIVATION: Alignment of medical images is a vital component of a large number of applications throughout the clinical track of events; not only within clinical diagnostic settings, but prominently so in the area of planning, consummation and evaluation of surgical and radiotherapeutical procedures. However, image registration of medical images is challenging because of variations on data appearance, imaging artifacts and complex data deformation problems. Hence, the aim of this study is to develop a robust image alignment method for medical images. RESULTS: An improved image registration method is proposed, and the method is evaluated with two types of medical data, including biological microscopic tissue images and dental X-ray images and compared with five state-of-the-art image registration techniques. The experimental results show that the presented method consistently performs well on both types of medical images, achieving 88.44 and 88.93% averaged registration accuracies for biological tissue images and X-ray images, respectively, and outperforms the benchmark methods. Based on the Tukey's honestly significant difference test and Fisher's least square difference test tests, the presented method performs significantly better than all existing methods (P ≤ 0.001) for tissue image alignment, and for the X-ray image registration, the proposed method performs significantly better than the two benchmark b-spline approaches (P < 0.001). AVAILABILITY: The software implementation of the presented method and the data used in this study are made publicly available for scientific communities to use (http://www-o.ntust.edu.tw/∼cweiwang/ImprovedImageRegistration/). CONTACT: cweiwang@mail.ntust.edu.tw.

Deep learning to assess microsatellite instability directly from histopathological whole slide images in endometrial cancer.

Molecular classification, particularly microsatellite instability-high (MSI-H), has gained attention for immunotherapy in endometrial cancer (EC). MSI-H is associated with DNA mismatch repair defects and is a crucial treatment predictor. The NCCN guidelines recommend pembrolizumab and nivolumab for advanced or recurrent MSI-H/mismatch repair deficient (dMMR) EC. However, evaluating MSI in all cases is impractical due to time and cost constraints. To overcome this challenge, we present an effective and efficient deep learning-based model designed to accurately and rapidly assess MSI status of EC using H&E-stained whole slide images. Our framework was evaluated on a comprehensive dataset of gigapixel histopathology images of 529 patients from the Cancer Genome Atlas (TCGA). The experimental results have shown that the proposed method achieved excellent performances in assessing MSI status, obtaining remarkably high results with 96%, 94%, 93% and 100% for endometrioid carcinoma G1G2, respectively, and 87%, 84%, 81% and 94% for endometrioid carcinoma G3, in terms of F-measure, accuracy, precision and sensitivity, respectively. Furthermore, the proposed deep learning framework outperforms four state-of-the-art benchmarked methods by a significant margin (p < 0.001) in terms of accuracy, precision, sensitivity and F-measure, respectively. Additionally, a run time analysis demonstrates that the proposed method achieves excellent quantitative results with high efficiency in AI inference time (1.03 seconds per slide), making the proposed framework viable for practical clinical usage. These results highlight the efficacy and efficiency of the proposed model to assess MSI status of EC directly from histopathological slides.

Computational Pathology for Breast Cancer and Gynecologic Cancer.

Advances in computation pathology have continued at an impressive pace in recent years [...].

Efficient Convolution Network to Assist Breast Cancer Diagnosis and Target Therapy.

Breast cancer is the leading cause of cancer-related deaths among women worldwide, and early detection and treatment has been shown to significantly reduce fatality rates from severe illness. Moreover, determination of the human epidermal growth factor receptor-2 (HER2) gene amplification by Fluorescence in situ hybridization (FISH) and Dual in situ hybridization (DISH) is critical for the selection of appropriate breast cancer patients for HER2-targeted therapy. However, visual examination of microscopy is time-consuming, subjective and poorly reproducible due to high inter-observer variability among pathologists and cytopathologists. The lack of consistency in identifying carcinoma-like nuclei has led to divergences in the calculation of sensitivity and specificity. This manuscript introduces a highly efficient deep learning method with low computing cost. The experimental results demonstrate that the proposed framework achieves high precision and recall on three essential clinical applications, including breast cancer diagnosis and human epidermal receptor factor 2 (HER2) amplification detection on FISH and DISH slides for HER2 target therapy. Furthermore, the proposed method outperforms the majority of the benchmark methods in terms of IoU by a significant margin (p<0.001) on three essential clinical applications. Importantly, run time analysis shows that the proposed method obtains excellent segmentation results with notably reduced time for Artificial intelligence (AI) training (16.93%), AI inference (17.25%) and memory usage (18.52%), making the proposed framework feasible for practical clinical usage.

Weakly supervised bilayer convolutional network in segmentation of HER2 related cells to guide HER2 targeted therapies.

Overexpression of human epidermal growth factor receptor 2 (HER2/ERBB2) is identified as a prognostic marker in metastatic breast cancer and a predictor to determine the effects of ERBB2-targeted drugs. Accurate ERBB2 testing is essential in determining the optimal treatment for metastatic breast cancer patients. Brightfield dual in situ hybridization (DISH) was recently authorized by the United States Food and Drug Administration for the assessment of ERRB2 overexpression, which however is a challenging task due to a variety of reasons. Firstly, the presence of touching clustered and overlapping cells render it difficult for segmentation of individual HER2 related cells, which must contain both ERBB2 and CEN17 signals. Secondly, the fuzzy cell boundaries make the localization of each HER2 related cell challenging. Thirdly, variation in the appearance of HER2 related cells is large. Fourthly, as manual annotations are usually made on targets with high confidence, causing sparsely labeled data with some unlabeled HER2 related cells defined as background, this will seriously confuse fully supervised AI learning and cause poor model outcomes. To deal with all issues mentioned above, we propose a two-stage weakly supervised deep learning framework for accurate and robust assessment of ERBB2 overexpression. The effectiveness and robustness of the proposed deep learning framework is evaluated on two DISH datasets acquired at two different magnifications. The experimental results demonstrate that the proposed deep learning framework achieves an accuracy of 96.78 ± 1.25, precision of 97.77 ± 3.09, recall of 84.86 ± 5.83 and Dice Index of 90.77 ± 4.1 and an accuracy of 96.43 ± 2.67, precision of 97.82 ± 3.99, recall of 87.14 ± 10.17 and Dice Index of 91.87 ± 6.51 for segmentation of ERBB2 overexpression on the two experimental datasets, respectively. Furthermore, the proposed deep learning framework outperforms 15 state-of-the-art benchmarked methods by a significant margin (P<0.05) with respect to IoU on both datasets.

Detection of ERBB2 and CEN17 signals in fluorescent in situ hybridization and dual in situ hybridization for guiding breast cancer HER2 target therapy.

The overexpression of the human epidermal growth factor receptor 2 (HER2) is a predictive biomarker in therapeutic effects for metastatic breast cancer. Accurate HER2 testing is critical for determining the most suitable treatment for patients. Fluorescent in situ hybridization (FISH) and dual in situ hybridization (DISH) have been recognized as FDA-approved methods to determine HER2 overexpression. However, analysis of HER2 overexpression is challenging. Firstly, the boundaries of cells are often unclear and blurry, with large variations in cell shapes and signals, making it challenging to identify the precise areas of HER2-related cells. Secondly, the use of sparsely labeled data, where some unlabeled HER2-related cells are classified as background, can significantly confuse fully supervised AI learning and result in unsatisfactory model outcomes. In this study, we present a weakly supervised Cascade R-CNN (W-CRCNN) model to automatically detect HER2 overexpression in HER2 DISH and FISH images acquired from clinical breast cancer samples. The experimental results demonstrate that the proposed W-CRCNN achieves excellent results in identification of HER2 amplification in three datasets, including two DISH datasets and a FISH dataset. For the FISH dataset, the proposed W-CRCNN achieves an accuracy of 0.970±0.022, precision of 0.974±0.028, recall of 0.917±0.065, F1-score of 0.943±0.042 and Jaccard Index of 0.899±0.073. For DISH datasets, the proposed W-CRCNN achieves an accuracy of 0.971±0.024, precision of 0.969±0.015, recall of 0.925±0.020, F1-score of 0.947±0.036 and Jaccard Index of 0.884±0.103 for dataset 1, and an accuracy of 0.978±0.011, precision of 0.975±0.011, recall of 0.918±0.038, F1-score of 0.946±0.030 and Jaccard Index of 0.884±0.052 for dataset 2, respectively. In comparison with the benchmark methods, the proposed W-CRCNN significantly outperforms all the benchmark approaches in identification of HER2 overexpression in FISH and DISH datasets (p<0.05). With the high degree of accuracy, precision and recall , the results show that the proposed method in DISH analysis for assessment of HER2 overexpression in breast cancer patients has significant potential to assist precision medicine.

Potential of methacrylated acemannan for exerting antioxidant-, cell proliferation-, and cell migration-inducing activities in vitro.

BACKGROUND: Acemannan is an acetylated polysaccharide of Aloe vera extract with antimicrobial, antitumor, antiviral, and antioxidant activities. This study aims to optimize the synthesis of acemannan from methacrylate powder using a simple method and characterize it for potential use as a wound-healing agent. METHODS: Acemannan was purified from methacrylated acemannan and characterized using high-performance liquid chromatography (HPLC), Fourier-transform infrared spectroscopy (FTIR), and 1H-nuclear magnetic resonance (NMR). 2,2-diphenyl-1-picrylhydrazyl (DPPH) and 3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyl-2H-tetrazolium bromide (MTT) assays were performed to investigate the antioxidant activity of acemannan and its effects on cell proliferation and oxidative stress damage, respectively. Further, a migration assay was conducted to determine the wound healing properties of acemannan. RESULTS: We successfully optimized the synthesis of acemannan from methacrylate powder using a simple method. Our results demonstrated that methacrylated acemannan was identified as a polysaccharide with an acetylation degree similar to that in A. vera, with the FTIR revealing peaks at 1739.94 cm-1 (C = O stretching vibration), 1370 cm-1 (deformation of the H-C-OH bonds), and 1370 cm-1 (C-O-C asymmetric stretching vibration); 1H NMR showed an acetylation degree of 1.202. The DPPH results showed the highest antioxidant activity of acemannan with a 45% radical clearance rate, compared to malvidin, CoQ10, and water. Moreover, 2000 µg/mL acemannan showed the most optimal concentration for inducing cell proliferation, while 5 µg/mL acemannan induced the highest cell migration after 3 h. In addition, MTT assay findings showed that after 24 h, acemannan treatment successfully recovered cell damage due to H2O2 pre-treatment. CONCLUSION: Our study provides a suitable technique for effective acemannan production and presents acemannan as a potential agent for use in accelerating wound healing through its antioxidant properties, as well as cell proliferation- and migration-inducing activities.

Interpretable attention-based deep learning ensemble for personalized ovarian cancer treatment without manual annotations.

Inhibition of pathological angiogenesis has become one of the first FDA approved targeted therapies widely tested in anti-cancer treatment, i.e. VEGF-targeting monoclonal antibody bevacizumab, in combination with chemotherapy for frontline and maintenance therapy for women with newly diagnosed ovarian cancer. Identification of the best predictive biomarkers of bevacizumab response is necessary in order to select patients most likely to benefit from this therapy. Hence, this study investigates the protein expression patterns on immunohistochemical whole slide images of three angiogenesis related proteins, including Vascular endothelial growth factor, Angiopoietin 2 and Pyruvate kinase isoform M2, and develops an interpretable and annotation-free attention based deep learning ensemble framework to predict the bevacizumab therapeutic effect on patients with epithelial ovarian cancer or peritoneal serous papillary carcinoma using tissue microarrays (TMAs). In evaluation with five-fold cross validation, the proposed ensemble model using the protein expressions of both Pyruvate kinase isoform M2 and Angiopoietin 2 achieves a notably high F-score (0.99±0.02), accuracy (0.99±0.03), precision (0.99±0.02), recall (0.99±0.02) and AUC (1.00±0). Kaplan-Meier progression free survival analysis confirms that the proposed ensemble is able to identify patients in the predictive therapeutic sensitive group with low cancer recurrence (p<0.001), and the Cox proportional hazards model analysis further confirms the above statement (p=0.012). In conclusion, the experimental results demonstrate that the proposed ensemble model using the protein expressions of both Pyruvate kinase isoform M2 and Angiopoietin 2 can assist treatment planning of bevacizumab targeted therapy for patients with ovarian cancer.

Fast automatic quantitative cell replication with fluorescent live cell imaging.

BACKGROUND: live cell imaging is a useful tool to monitor cellular activities in living systems. It is often necessary in cancer research or experimental research to quantify the dividing capabilities of cells or the cell proliferation level when investigating manipulations of the cells or their environment. Manual quantification of fluorescence microscopic image is difficult because human is neither sensitive to fine differences in color intensity nor effective to count and average fluorescence level among cells. However, auto-quantification is not a straightforward problem to solve. As the sampling location of the microscopy changes, the amount of cells in individual microscopic images varies, which makes simple measurement methods such as the sum of stain intensity values or the total number of positive stain within each image inapplicable. Thus, automated quantification with robust cell segmentation techniques is required. RESULTS: An automated quantification system with robust cell segmentation technique are presented. The experimental results in application to monitor cellular replication activities show that the quantitative score is promising to represent the cell replication level, and scores for images from different cell replication groups are demonstrated to be statistically significantly different using ANOVA, LSD and Tukey HSD tests (p-value < 0.01). In addition, the technique is fast and takes less than 0.5 second for high resolution microscopic images (with image dimension 2560 × 1920). CONCLUSION: A robust automated quantification method of live cell imaging is built to measure the cell replication level, providing a robust quantitative analysis system in fluorescent live cell imaging. In addition, the presented unsupervised entropy based cell segmentation for live cell images is demonstrated to be also applicable for nuclear segmentation of IHC tissue images.

A Survey on Deep Learning for Precision Oncology.

Precision oncology, which ensures optimized cancer treatment tailored to the unique biology of a patient's disease, has rapidly developed and is of great clinical importance. Deep learning has become the main method for precision oncology. This paper summarizes the recent deep-learning approaches relevant to precision oncology and reviews over 150 articles within the last six years. First, we survey the deep-learning approaches categorized by various precision oncology tasks, including the estimation of dose distribution for treatment planning, survival analysis and risk estimation after treatment, prediction of treatment response, and patient selection for treatment planning. Secondly, we provide an overview of the studies per anatomical area, including the brain, bladder, breast, bone, cervix, esophagus, gastric, head and neck, kidneys, liver, lung, pancreas, pelvis, prostate, and rectum. Finally, we highlight the challenges and discuss potential solutions for future research directions.

Fast Segmentation of Metastatic Foci in H&E Whole-Slide Images for Breast Cancer Diagnosis.

Breast cancer is the leading cause of death for women globally. In clinical practice, pathologists visually scan over enormous amounts of gigapixel microscopic tissue slide images, which is a tedious and challenging task. In breast cancer diagnosis, micro-metastases and especially isolated tumor cells are extremely difficult to detect and are easily neglected because tiny metastatic foci might be missed in visual examinations by medical doctors. However, the literature poorly explores the detection of isolated tumor cells, which could be recognized as a viable marker to determine the prognosis for T1NoMo breast cancer patients. To address these issues, we present a deep learning-based framework for efficient and robust lymph node metastasis segmentation in routinely used histopathological hematoxylin−eosin-stained (H−E) whole-slide images (WSI) in minutes, and a quantitative evaluation is conducted using 188 WSIs, containing 94 pairs of H−E-stained WSIs and immunohistochemical CK(AE1/AE3)-stained WSIs, which are used to produce a reliable and objective reference standard. The quantitative results demonstrate that the proposed method achieves 89.6% precision, 83.8% recall, 84.4% F1-score, and 74.9% mIoU, and that it performs significantly better than eight deep learning approaches, including two recently published models (v3_DCNN and Xception-65), and three variants of Deeplabv3+ with three different backbones, namely, U-Net, SegNet, and FCN, in precision, recall, F1-score, and mIoU (p<0.001). Importantly, the proposed system is shown to be capable of identifying tiny metastatic foci in challenging cases, for which there are high probabilities of misdiagnosis in visual inspection, while the baseline approaches tend to fail in detecting tiny metastatic foci. For computational time comparison, the proposed method takes 2.4 min for processing a WSI utilizing four NVIDIA Geforce GTX 1080Ti GPU cards and 9.6 min using a single NVIDIA Geforce GTX 1080Ti GPU card, and is notably faster than the baseline methods (4-times faster than U-Net and SegNet, 5-times faster than FCN, 2-times faster than the 3 different variants of Deeplabv3+, 1.4-times faster than v3_DCNN, and 41-times faster than Xception-65).

Multigenerational inspections of environmental thermal perturbations promote metabolic trade-offs in developmental stages of tropical fish.

Global warming both reduces global temperature variance and increases the frequency of extreme weather events. In response to these ambient perturbations, animals may be subject to trans- or intra-generational phenotype modifications that help to maintain homeostasis and fitness. Here, we show how temperature-associated transgenerational plasticity in tilapia affects metabolic trade-offs during developmental stages under a global warming scenario. Tropical tilapia reared at a stable temperature of 27 °C for a decade were divided into two temperature-experience groups for four generations of breeding. Each generation of one group was exposed to a single 15 °C cold-shock experience during its lifetime (cold-experienced CE group), and the other group was kept stably at 27 °C throughout their lifetimes (cold-naïve CN group). The offspring at early life stages from the CE and CN tilapia were then assessed by metabolomics-based profiling, and the results implied that parental cold-experience might affect energy provision during reproduction. Furthermore, at early life stages, progeny may be endowed with metabolic traits that help the animals cope with ambient temperature perturbations. This study also applied the feature rescaling and Uniform Manifold Approximation and Projection (UMAP) to visualize metabolic dynamics, and the result could effectively decompose the complex omic-based datasets to represent the energy trade-off variability. For example, the carbohydrate to free amino acid conversion and enhanced compensatory features appeared to be hypothermic-responsive traits. These multigenerational metabolic effects suggest that the tropical ectothermic tilapia may exhibit transgenerational phenotype plasticity, which could optimize energy allocation under ambient temperature challenges. Knowledge about such metabolism-related transgenerational plasticity effects in ectothermic aquatic species may allow us to better predict how adaptive mechanisms will affect fish populations in a climate with narrow temperature variation and frequent extreme weather events.

Fast cross-staining alignment of gigapixel whole slide images with application to prostate cancer and breast cancer analysis.

Joint analysis of multiple protein expressions and tissue morphology patterns is important for disease diagnosis, treatment planning, and drug development, requiring cross-staining alignment of multiple immunohistochemical and histopathological slides. However, cross-staining alignment of enormous gigapixel whole slide images (WSIs) at single cell precision is difficult. Apart from gigantic data dimensions of WSIs, there are large variations on the cell appearance and tissue morphology across different staining together with morphological deformations caused by slide preparation. The goal of this study is to build an image registration framework for cross-staining alignment of gigapixel WSIs of histopathological and immunohistochemical microscopic slides and assess its clinical applicability. To the authors' best knowledge, this is the first study to perform real time fully automatic cross staining alignment of WSIs with 40× and 20× objective magnification. The proposed WSI registration framework consists of a rapid global image registration module, a real time interactive field of view (FOV) localization model and a real time propagated multi-level image registration module. In this study, the proposed method is evaluated on two kinds of cancer datasets from two hospitals using different digital scanners, including a dual staining breast cancer data set with 43 hematoxylin and eosin (H&E) WSIs and 43 immunohistochemical (IHC) CK(AE1/AE3) WSIs, and a triple staining prostate cancer data set containing 30 H&E WSIs, 30 IHC CK18 WSIs, and 30 IHC HMCK WSIs. In evaluation, the registration performance is measured by not only registration accuracy but also computational time. The results show that the proposed method achieves high accuracy of 0.833 ± 0.0674 for the triple-staining prostate cancer data set and 0.931 ± 0.0455 for the dual-staining breast cancer data set, respectively, and takes only 4.34 s per WSI registration on average. In addition, for 30.23% data, the proposed method takes less than 1 s for WSI registration. In comparison with the benchmark methods, the proposed method demonstrates superior performance in registration accuracy and computational time, which has great potentials for assisting medical doctors to identify cancerous tissues and determine the cancer stage in clinical practice.