ABSTRACT
INTRODUCTION: Amyloidosis, including cerebral amyloid angiopathy, and markers of small vessel disease (SVD) vary across dominantly inherited Alzheimer's disease (DIAD) presenilin-1 (PSEN1) mutation carriers. We investigated how mutation position relative to codon 200 (pre-/postcodon 200) influences these pathologic features and dementia at different stages. METHODS: Individuals from families with known PSEN1 mutations (n = 393) underwent neuroimaging and clinical assessments. We cross-sectionally evaluated regional Pittsburgh compound B-positron emission tomography uptake, magnetic resonance imaging markers of SVD (diffusion tensor imaging-based white matter injury, white matter hyperintensity volumes, and microhemorrhages), and cognition. RESULTS: Postcodon 200 carriers had lower amyloid burden in all regions but worse markers of SVD and worse Clinical Dementia Rating® scores compared to precodon 200 carriers as a function of estimated years to symptom onset. Markers of SVD partially mediated the mutation position effects on clinical measures. DISCUSSION: We demonstrated the genotypic variability behind spatiotemporal amyloidosis, SVD, and clinical presentation in DIAD, which may inform patient prognosis and clinical trials. HIGHLIGHTS: Mutation position influences Aß burden, SVD, and dementia. PSEN1 pre-200 group had stronger associations between Aß burden and disease stage. PSEN1 post-200 group had stronger associations between SVD markers and disease stage. PSEN1 post-200 group had worse dementia score than pre-200 in late disease stage. Diffusion tensor imaging-based SVD markers mediated mutation position effects on dementia in the late stage.
Subject(s)
Alzheimer Disease , Amyloidosis , Cerebral Small Vessel Diseases , Humans , Alzheimer Disease/diagnostic imaging , Alzheimer Disease/genetics , Alzheimer Disease/pathology , Cerebral Small Vessel Diseases/diagnostic imaging , Cerebral Small Vessel Diseases/genetics , Cerebral Small Vessel Diseases/complications , Diffusion Tensor Imaging , Magnetic Resonance Imaging , Mutation/genetics , Presenilin-1/geneticsABSTRACT
BACKGROUND: Robotic thoracic surgery is a minimally invasive technique that allows the surgeon to perform delicate, accurate surgical manoeuvres within the chest cavity without rib spreading. Previous studies have suggested potential benefits of the robotic platform in nodal upstaging due to its versatility, seven degrees of freedom of movement, and superior vision. However, there is currently a paucity of robust clinical data from Australia. AIMS: This study aimed to assess the perioperative safety and oncological efficacy of anatomical pulmonary resections performed using the robotic platform. Endpoints included mortality and major morbidity outcomes according to Clavien-Dindo classification and rate of pathological nodal upstaging compared with preoperative imaging using positron emission tomography. METHODS: A single-surgeon retrospective analysis was performed using data collected from two institutions from July 2021 to May 2022, after ethics committee approval. Consecutive patients who underwent anatomical robotic resections were included in the study, with subsequent analysis of patients who had confirmed primary lung cancer. RESULTS: A total of 52 patients underwent robotic anatomical pulmonary resection during the study period. Safety was demonstrated by 0% mortality and a 9.6% major complication rate, which was related to chest tube insertion for prolonged air leak or intensive care unit monitoring during treatment of atrial arrhythmia. After excluding patients who did not have primary lung cancer, 48 patients remained for further analysis; pathological nodal upstaging was observed in nine (18.8%) of these patients. On multivariate analysis, the total number of lymph nodes harvested was found to be a statistically significant predictor of nodal upstaging. Complete microscopic resection (R0) was achieved in 100% of patients. CONCLUSIONS: This study represents the most extensive documentation of robotic thoracic procedures in Australia in the existing literature. It demonstrated a satisfactory safety profile with a relatively high rate of nodal upstaging, possibly reflecting the ability of the robotic instruments to perform comprehensive and complete nodal resection at the time of anatomical pulmonary resection.
Subject(s)
Carcinoma, Non-Small-Cell Lung , Lung Neoplasms , Robotic Surgical Procedures , Humans , Carcinoma, Non-Small-Cell Lung/pathology , Carcinoma, Non-Small-Cell Lung/surgery , Lung Neoplasms/surgery , Robotic Surgical Procedures/methods , Retrospective Studies , Pneumonectomy/methods , Neoplasm Staging , Australia/epidemiology , Tomography, X-Ray ComputedABSTRACT
[18F]-Fluorodeoxyglucose (FDG) positron emission tomography - computed tomography (PET-CT) has become the imaging modality of choice for diagnosing many cancers. Co-learning complementary PET-CT imaging features is a fundamental requirement for automatic tumor segmentation and for developing computer aided cancer diagnosis systems. In this study, we propose a hyper-connected transformer (HCT) network that integrates a transformer network (TN) with a hyper connected fusion for multi-modality PET-CT images. The TN was leveraged for its ability to provide global dependencies in image feature learning, which was achieved by using image patch embeddings with a self-attention mechanism to capture image-wide contextual information. We extended the single-modality definition of TN with multiple TN based branches to separately extract image features. We also introduced a hyper connected fusion to fuse the contextual and complementary image features across multiple transformers in an iterative manner. Our results with two clinical datasets show that HCT achieved better performance in segmentation accuracy when compared to the existing methods.Clinical Relevance-We anticipate that our approach can be an effective and supportive tool to aid physicians in tumor quantification and in identifying image biomarkers for cancer treatment.
Subject(s)
Neoplasms , Positron Emission Tomography Computed Tomography , Humans , Positron Emission Tomography Computed Tomography/methods , Neoplasms/diagnostic imaging , Image Interpretation, Computer-Assisted/methods , Fluorodeoxyglucose F18 , Diagnosis, Computer-AssistedABSTRACT
Carriers of mutations responsible for dominantly inherited Alzheimer disease provide a unique opportunity to study potential imaging biomarkers. Biomarkers based on routinely acquired clinical MR images, could supplement the extant invasive or logistically challenging) biomarker studies. We used 1104 longitudinal MR, 324 amyloid beta, and 87 tau positron emission tomography imaging sessions from 525 participants enrolled in the Dominantly Inherited Alzheimer Network Observational Study to extract novel imaging metrics representing the mean (µ) and standard deviation (σ) of standardized image intensities of T1-weighted and Fluid attenuated inversion recovery (FLAIR) MR scans. There was an exponential decrease in FLAIR-µ in mutation carriers and an increase in FLAIR and T1 signal heterogeneity (T1-σ and FLAIR-σ) as participants approached the symptom onset in both supramarginal, the right postcentral and right superior temporal gyri as well as both caudate nuclei, putamina, thalami, and amygdalae. After controlling for the effect of regional atrophy, FLAIR-µ decreased and T1-σ and FLAIR-σ increased with increasing amyloid beta and tau deposition in numerous cortical regions. In symptomatic mutation carriers and independent of the effect of regional atrophy, tau pathology demonstrated a stronger relationship with image intensity metrics, compared with amyloid pathology. We propose novel MR imaging intensity-based metrics using standard clinical T1 and FLAIR images which strongly associates with the progression of pathology in dominantly inherited Alzheimer disease. We suggest that tau pathology may be a key driver of the observed changes in this cohort of patients.
Subject(s)
Alzheimer Disease , Humans , Alzheimer Disease/diagnostic imaging , Alzheimer Disease/genetics , Alzheimer Disease/complications , Amyloid beta-Peptides , Magnetic Resonance Imaging/methods , Positron-Emission Tomography , Biomarkers , Atrophy , tau ProteinsABSTRACT
OBJECTIVE: We aimed to (i) compare the clinical, neuropsychological, and neuroimaging characteristics of unprovoked late-onset epilepsy (LOE) patients with cognitive symptoms against probable Alzheimer's disease (AD) patients; (ii) clarify how neurodegeneration and other processes could be implicated in the cognitive symptoms of unprovoked LOE patients; and (iii) characterize the longitudinal trajectory of unprovoked LOE patients with cognitive symptoms. METHODS: Twenty-six unprovoked LOE patients with cognitive symptoms and 26 probable AD were retrospectively recruited from epilepsy and memory clinics at a single tertiary referral center. The patients underwent comprehensive clinical, neuropsychological, and 18Fluorodeoxyglucose PET-CT assessments. All LOE patients had clinical follow-up and a subset of 17 patients had repeat neuropsychological assessments. RESULTS: At baseline, 18% of LOE patients with cognitive symptoms had dementia-range cognitive impairment and one received a diagnosis of probable AD. Compared with the probable AD group, the LOE group did not perform significantly better in global measures of cognition (total ACE-III), neuropsychological tests for fluency, working memory, language, attention, or executive function, but performed better in naming, memory, and visuospatial ability. The commonest patterns of cognitive impairment in the LOE group were frontal and left temporal, whereas all AD patients exhibited parietotemporal patterns. The AD group had more 18Fluorodeoxyglucose PET-CT hypometabolism in the parietal and occipital, but not the temporal and frontal lobes. During the 3.0 ± 3.2 years follow-up, improved seizure frequency in the LOE group covaried with improved total ACE-III score, there was no further conversion to probable AD and no group-level cognitive decline. CONCLUSION: Unprovoked LOE patients with cognitive symptoms had varying severities of cognitive impairment, and different patterns of cognitive and imaging abnormalities compared with AD patients. They were rarely diagnosed with probable AD at presentation or follow-up. Cognitive outcome in LOE may be related to seizure control. Cerebral small vessel disease may play a role in LOE-associated cognitive impairment.
Subject(s)
Alzheimer Disease , Cognitive Dysfunction , Epilepsy , Humans , Alzheimer Disease/complications , Alzheimer Disease/diagnostic imaging , Positron Emission Tomography Computed Tomography , Retrospective Studies , Cognition , Cognitive Dysfunction/complications , Cognitive Dysfunction/diagnostic imaging , Positron-Emission Tomography/methods , Neuropsychological Tests , Fluorodeoxyglucose F18 , Epilepsy/complications , Epilepsy/diagnostic imaging , SeizuresABSTRACT
Coronavirus disease 2019 (COVID-19) is caused by Severe Acute Respiratory Syndrome Coronavirus 2 which enters the body via the angiotensin-converting enzyme 2 (ACE2) and altering its gene expression. Altered ACE2 plays a crucial role in the pathogenesis of COVID-19. Gene expression profiling, however, is invasive and costly, and is not routinely performed. In contrast, medical imaging such as computed tomography (CT) captures imaging features that depict abnormalities, and it is widely available. Computerized quantification of image features has enabled 'radiogenomics', a research discipline that identifies image features that are associated with molecular characteristics. Radiogenomics between ACE2 and COVID-19 has yet to be done primarily due to the lack of ACE2 expression data among COVID-19 patients. Similar to COVID-19, patients with lung adenocarcinoma (LUAD) exhibit altered ACE2 expression and, LUAD data are abundant. We present a radiogenomics framework to derive image features (ACE2-RGF) associated with ACE2 expression data from LUAD. The ACE2-RGF was then used as a surrogate biomarker for ACE2 expression. We adopted conventional feature selection techniques including ElasticNet and LASSO. Our results show that: i) the ACE2-RGF encoded a distinct collection of image features when compared to conventional techniques, ii) the ACE2-RGF can classify COVID-19 from normal subjects with a comparable performance to conventional feature selection techniques with an AUC of 0.92, iii) ACE2-RGF can effectively identify patients with critical illness with an AUC of 0.85. These findings provide unique insights for automated COVID-19 analysis and future research.
Subject(s)
COVID-19 , Humans , COVID-19/diagnostic imaging , Angiotensin-Converting Enzyme 2 , Peptidyl-Dipeptidase A/genetics , Peptidyl-Dipeptidase A/metabolism , SARS-CoV-2/metabolism , Tomography, X-Ray ComputedABSTRACT
The Dominantly Inherited Alzheimer Network (DIAN) is an international collaboration studying autosomal dominant Alzheimer disease (ADAD). ADAD arises from mutations occurring in three genes. Offspring from ADAD families have a 50% chance of inheriting their familial mutation, so non-carrier siblings can be recruited for comparisons in case-control studies. The age of onset in ADAD is highly predictable within families, allowing researchers to estimate an individual's point in the disease trajectory. These characteristics allow candidate AD biomarker measurements to be reliably mapped during the preclinical phase. Although ADAD represents a small proportion of AD cases, understanding neuroimaging-based changes that occur during the preclinical period may provide insight into early disease stages of 'sporadic' AD also. Additionally, this study provides rich data for research in healthy aging through inclusion of the non-carrier controls. Here we introduce the neuroimaging dataset collected and describe how this resource can be used by a range of researchers.
Subject(s)
Alzheimer Disease , Arthrogryposis , Humans , Alzheimer Disease/diagnostic imaging , Alzheimer Disease/genetics , Positron-Emission Tomography , Magnetic Resonance Imaging , Neuroimaging , Mutation/genetics , Amyloid beta-Peptides/geneticsABSTRACT
The pyridinyl-butadienyl-benzothiazole (PBB3 15) scaffold was used to develop tau ligands with improved in vitro and in vivo properties for imaging applications to provide insights into the etiology and characteristics of Alzheimer's disease. The photoisomerisable trans-butadiene bridge of PBB3 was replaced with 1,2,3-triazole, amide, and ester moieties and in vitro fluorescence staining studies revealed that triazole derivatives showed good visualisation of Aß plaques, but failed to detect the neurofibrillary tangles (NFTs) in human brain sections. However, NFTs could be observed using the amide 110 and ester 129. Furthermore, the ligands showed low to high affinities (Ki = >1.5 mM-0.46 nM) at the shared binding site(s) with PBB3.
ABSTRACT
We present a case of semantic variant primary progressive aphasia as the presenting feature in a patient with Huntington disease (HD). The patient initially developed progressive language impairment including impaired naming and object knowledge and single-word comprehension and then developed chorea and behavioral changes. An MRI of the brain showed left anterior temporal lobe and hippocampal atrophy. A neurologic FDG PET/CT showed reduced metabolism in the head of the left caudate nucleus. Huntingtin gene testing revealed an expansion of 39 CAG repeats in 1 allele. This case outlines the substantial overlap between the clinical presentation of HD and frontotemporal lobar degeneration syndromes and provides commentary on the investigation of these neurodegenerative diseases.
Subject(s)
Aphasia, Primary Progressive , Huntington Disease , Neurodegenerative Diseases , Humans , Semantics , Aphasia, Primary Progressive/diagnostic imaging , Aphasia, Primary Progressive/etiology , Huntington Disease/diagnosis , Huntington Disease/diagnostic imaging , Positron Emission Tomography Computed Tomography , Brain/diagnostic imaging , Magnetic Resonance ImagingABSTRACT
INTRODUCTION: As the number of biomarkers used to study Alzheimer's disease (AD) continues to increase, it is important to understand the utility of any given biomarker, as well as what additional information a biomarker provides when compared to others. METHODS: We used hierarchical clustering to group 19 cross-sectional biomarkers in autosomal dominant AD. Feature selection identified biomarkers that were the strongest predictors of mutation status and estimated years from symptom onset (EYO). Biomarkers identified included clinical assessments, neuroimaging, cerebrospinal fluid amyloid, and tau, and emerging biomarkers of neuronal integrity and inflammation. RESULTS: Three primary clusters were identified: neurodegeneration, amyloid/tau, and emerging biomarkers. Feature selection identified amyloid and tau measures as the primary predictors of mutation status and EYO. Emerging biomarkers of neuronal integrity and inflammation were relatively weak predictors. DISCUSSION: These results provide novel insight into our understanding of the relationships among biomarkers and the staging of biomarkers based on disease progression.
Subject(s)
Alzheimer Disease , Humans , Alzheimer Disease/diagnosis , Amyloid beta-Peptides/cerebrospinal fluid , Amyloidogenic Proteins , Biomarkers/cerebrospinal fluid , Cross-Sectional Studies , Inflammation , tau Proteins/genetics , tau Proteins/cerebrospinal fluidABSTRACT
Prostate cancer is the most common cancer and the second leading cause of cancer death in men. The imaging assessment and treatment of prostate cancer has vastly improved over the past decade. The introduction of PSMA PET-CT has improved the detection of loco-regional and metastatic disease. PSMA PET-CT also has a role in the primary diagnosis and staging, in detecting biochemical recurrence after curative treatment and in metastasis-directed therapy. In this paper we review the role of PSMA PET-CT in prostate cancer.
ABSTRACT
OBJECTIVE: The accurate classification of mass lesions in the adrenal glands ('adrenal masses'), detected with computed tomography (CT), is important for diagnosis and patient management. Adrenal masses can be benign or malignant and benign masses have varying prevalence. Classification methods based on convolutional neural networks (CNNs) are the state-of-the-art in maximizing inter-class differences in large medical imaging training datasets. The application of CNNs, to adrenal masses is challenging due to large intra-class variations, large inter-class similarities and imbalanced training data due to the size of the mass lesions. METHODS: We developed a deep multi-scale resemblance network (DMRN) to overcome these limitations and leveraged paired CNNs to evaluate the intra-class similarities. We used multi-scale feature embedding to improve the inter-class separability by iteratively combining complementary information produced at different scales of the input to create structured feature descriptors. We augmented the training data with randomly sampled paired adrenal masses to reduce the influence of imbalanced training data. RESULTS: We used 229 CT scans of patients with adrenal masses for evaluation. In a five-fold cross-validation, our method had the best results (89.52 % in accuracy) when compared to the state-of-the-art methods (p < 0.05). We conducted a generalizability analysis of our method on the ImageCLEF 2016 competition dataset for medical subfigure classification, which consists of a training set of 6776 images and a test set of 4166 images across 30 classes. Our method achieved better classification performance (85.90 % in accuracy) when compared to the existing methods and was competitive when compared with methods that require additional training data (1.47 % lower in accuracy). CONCLUSION: Our DMRN sub-classified adrenal masses on CT and was superior to state-of-the-art approaches.
Subject(s)
Neural Networks, Computer , Tomography, X-Ray Computed , Humans , Tomography, X-Ray Computed/methodsABSTRACT
Deformable image registration is fundamental for many medical image analyses. A key obstacle for accurate image registration lies in image appearance variations such as the variations in texture, intensities, and noise. These variations are readily apparent in medical images, especially in brain images where registration is frequently used. Recently, deep learning-based registration methods (DLRs), using deep neural networks, have shown computational efficiency that is several orders of magnitude faster than traditional optimization-based registration methods (ORs). DLRs rely on a globally optimized network that is trained with a set of training samples to achieve faster registration. DLRs tend, however, to disregard the target-pair-specific optimization inherent in ORs and thus have degraded adaptability to variations in testing samples. This limitation is severe for registering medical images with large appearance variations, especially since few existing DLRs explicitly take into account appearance variations. In this study, we propose an Appearance Adjustment Network (AAN) to enhance the adaptability of DLRs to appearance variations. Our AAN, when integrated into a DLR, provides appearance transformations to reduce the appearance variations during registration. In addition, we propose an anatomy-constrained loss function through which our AAN generates anatomy-preserving transformations. Our AAN has been purposely designed to be readily inserted into a wide range of DLRs and can be trained cooperatively in an unsupervised and end-to-end manner. We evaluated our AAN with three state-of-the-art DLRs - Voxelmorph (VM), Diffeomorphic Voxelmorph (DifVM), and Laplacian Pyramid Image Registration Network (LapIRN) - on three well-established public datasets of 3D brain magnetic resonance imaging (MRI) - IBSR18, Mindboggle101, and LPBA40. The results show that our AAN consistently improved existing DLRs and outperformed state-of-the-art ORs on registration accuracy, while adding a fractional computational load to existing DLRs.
Subject(s)
Image Processing, Computer-Assisted , Magnetic Resonance Imaging , Algorithms , Brain/diagnostic imaging , Humans , Image Processing, Computer-Assisted/methods , Neural Networks, ComputerABSTRACT
The identification of melanoma involves an integrated analysis of skin lesion images acquired using clinical and dermoscopy modalities. Dermoscopic images provide a detailed view of the subsurface visual structures that supplement the macroscopic details from clinical images. Visual melanoma diagnosis is commonly based on the 7-point visual category checklist (7PC), which involves identifying specific characteristics of skin lesions. The 7PC contains intrinsic relationships between categories that can aid classification, such as shared features, correlations, and the contributions of categories towards diagnosis. Manual classification is subjective and prone to intra- and interobserver variability. This presents an opportunity for automated methods to aid in diagnostic decision support. Current state-of-the-art methods focus on a single image modality (either clinical or dermoscopy) and ignore information from the other, or do not fully leverage the complementary information from both modalities. Furthermore, there is not a method to exploit the 'intercategory' relationships in the 7PC. In this study, we address these issues by proposing a graph-based intercategory and intermodality network (GIIN) with two modules. A graph-based relational module (GRM) leverages intercategorical relations, intermodal relations, and prioritises the visual structure details from dermoscopy by encoding category representations in a graph network. The category embedding learning module (CELM) captures representations that are specialised for each category and support the GRM. We show that our modules are effective at enhancing classification performance using three public datasets (7PC, ISIC 2017, and ISIC 2018), and that our method outperforms state-of-the-art methods at classifying the 7PC categories and diagnosis.
Subject(s)
Melanoma , Skin Diseases , Skin Neoplasms , Humans , Dermoscopy/methods , Skin Neoplasms/diagnostic imaging , Melanoma/diagnostic imagingABSTRACT
[68Ga]Ga-PSMA-11, a urea-based peptidomimetic, is a diagnostic radiopharmaceutical for positron emission tomography (PET) imaging that targets the prostate-specific membrane antigen (PSMA). The recent Food and Drug Administration approval of [68Ga]Ga-PSMA-11 for PET imaging of patients with prostate cancer, expected follow-up approval of companion radiotherapeutics (e.g., [177Lu]Lu-PSMA-617, [225Ac]Ac-PSMA-617) and large prostate cancer patient volumes requiring access are poised to create an unprecedented demand for [68Ga]Ga-PSMA-11 in nuclear medicine clinics around the world. Meeting this global demand is going to require a variety of synthesis methods compatible with 68Ga eluted from a generator or produced on a cyclotron. To address this urgent need in the PET radiochemistry community, herein we report detailed protocols for the synthesis of [68Ga]Ga-PSMA-11, (also known as HBED-CC, Glu-urea-Lys(Ahx)-HBED-CC and PSMA-HBED-CC) using both generator-eluted and cyclotron-produced 68Ga and contrast the pros and cons of each method. The radiosyntheses are automated and have been validated for human use at two sites (University of Michigan (UM), United States; Royal Prince Alfred Hospital (RPA), Australia) and used to produce [68Ga]Ga-PSMA-11 for patient use in good activity yields (single generator, 0.52 GBq (14 mCi); dual generators, 1.04-1.57 GBq (28-42 mCi); cyclotron method (single target), 1.47-1.89 GBq (40-51 mCi); cyclotron method (dual target), 3.63 GBq (98 mCi)) and high radiochemical purity (99%) (UM, n = 645; RPA, n > 600). Both methods are appropriate for clinical production but, in the long term, the method employing cyclotron-produced 68Ga is the most promising for meeting high patient volumes. Quality control testing (visual inspection, pH, radiochemical purity and identity, radionuclidic purity and identity, sterile filter integrity, bacterial endotoxin content, sterility, stability) confirmed doses are suitable for clinical use, and there is no difference in clinical prostate cancer PET imaging using [68Ga]Ga-PSMA-11 prepared using the two production methods.
Subject(s)
Prostatic Neoplasms , Radiopharmaceuticals , Cyclotrons , Edetic Acid , Gallium Radioisotopes/chemistry , Humans , Male , Positron-Emission Tomography/methods , Prostatic Neoplasms/diagnostic imaging , UreaABSTRACT
Radiogenomics relationships (RRs) aims to identify statistically significant correlations between medical image features and molecular characteristics from analysing tissue samples. Previous radiogenomics studies mainly relied on a single category of image feature extraction techniques (ETs); these are (i) handcrafted ETs that encompass visual imaging characteristics, curated from knowledge of human experts and, (ii) deep ETs that quantify abstract-level imaging characteristics from large data. Prior studies therefore failed to leverage the complementary information that are accessible from fusing the ETs. In this study, we propose a fused feature signature (FFSig): a selection of image features from handcrafted and deep ETs (e.g., transfer learning and fine-tuning of deep learning models). We evaluated the FFSig's ability to better represent RRs compared to individual ET approaches with two public datasets: the first dataset was used to build the FFSig using 89 patients with non-small cell lung cancer (NSCLC) comprising of gene expression data and CT images of the thorax and the upper abdomen for each patient; the second NSCLC dataset comprising of 117 patients with CT images and RNA-Seq data and was used as the validation set. Our results show that our FFSig encoded complementary imaging characteristics of tumours and identified more RRs with a broader range of genes that are related to important biological functions such as tumourigenesis. We suggest that the FFSig has the potential to identify important RRs that may assist cancer diagnosis and treatment in the future.
Subject(s)
Carcinoma, Non-Small-Cell Lung/diagnostic imaging , Carcinoma, Non-Small-Cell Lung/genetics , Imaging Genomics , Lung Neoplasms/diagnostic imaging , Lung Neoplasms/genetics , Deep Learning , Gene Ontology , Humans , RNA-Seq , Tomography, X-Ray Computed , TranscriptomeABSTRACT
Segmentation of skin lesions is an important step for imaging-based clinical decision support systems. Automatic skin lesion segmentation methods based on fully convolutional networks (FCNs) are regarded as the state-of-the-art for accuracy. When there are, however, insufficient training data to cover all the variations in skin lesions, where lesions from different patients may have major differences in size/shape/texture, these methods failed to segment the lesions that have image characteristics, which are less common in the training datasets. FCN-based semi-automatic segmentation methods, which fuse user-inputs with high-level semantic image features derived from FCNs offer an ideal complement to overcome limitations of automatic segmentation methods. These semi-automatic methods rely on the automated state-of-the-art FCNs coupled with user-inputs for refinements, and therefore being able to tackle challenging skin lesions. However, there are a limited number of FCN-based semi-automatic segmentation methods and all these methods focused on 'early-fusion', where the first few convolutional layers are used to fuse image features and user-inputs and then derive fused image features for segmentation. For early-fusion based methods, because the user-input information can be lost after the first few convolutional layers, consequently, the user-input information will have limited guidance and constraint in segmenting the challenging skin lesions with inhomogeneous textures and fuzzy boundaries. Hence, in this work, we introduce a hyper-fusion network (HFN) to fuse the extracted user-inputs and image features over multiple stages. We separately extract complementary features which then allows for an iterative use of user-inputs along all the fusion stages to refine the segmentation. We evaluated our HFN on three well-established public benchmark datasets - ISBI Skin Lesion Challenge 2017, 2016 and PH2 - and our results show that the HFN is more accurate and generalizable than the state-of-the-art methods, in particular with challenging skin lesions.