NeighborNet: Learning Intra- and Inter-Image Pixel Neighbor Representation for Breast Lesion Segmentation.

Breast lesion segmentation from ultrasound images is essential in computer-aided breast cancer diagnosis. To alleviate the problems of blurry lesion boundaries and irregular morphologies, common practices combine CNN and attention to integrate global and local information. However, previous methods use two independent modules to extract global and local features separately, such feature-wise inflexible integration ignores the semantic gap between them, resulting in representation redundancy/insufficiency and undesirable restrictions in clinic practices. Moreover, medical images are highly similar to each other due to the imaging methods and human tissues, but the captured global information by transformer-based methods in the medical domain is limited within images, the semantic relations and common knowledge across images are largely ignored. To alleviate the above problems, in the neighbor view, this paper develops a pixel neighbor representation learning method (NeighborNet) to flexibly integrate global and local context within and across images for lesion morphology and boundary modeling. Concretely, we design two neighbor layers to investigate two properties (i.e., number and distribution) of neighbors. The neighbor number for each pixel is not fixed but determined by itself. The neighbor distribution is extended from one image to all images in the datasets. With the two properties, for each pixel at each feature level, the proposed NeighborNet can evolve into the transformer or degenerate into the CNN for adaptive context representation learning to cope with the irregular lesion morphologies and blurry boundaries. The state-of-the-art performances on three ultrasound datasets prove the effectiveness of the proposed NeighborNet. The code is available at: https://github.com/fjcaoww/NeighborNet.

Dual-Spatial Domain Generalization for Fundus Lesion Segmentation in Unseen Manufacturer's OCT Images.

OBJECTIVE: Optical Coherence Tomography (OCT) images can provide non-invasive visualization of fundus lesions; however, scanners from different OCT manufacturers largely vary from each other, which often leads to model deterioration to unseen OCT scanners due to domain shift. METHODS: To produce the T-styles of the potential target domain, an Orthogonal Style Space Reparameterization (OSSR) method is proposed to apply orthogonal constraints in the latent orthogonal style space to the sampled marginal styles. To leverage the high-level features of multi-source domains and potential T-styles in the graph semantic space, a Graph Adversarial Network (GAN) is constructed to align the generated samples with the source domain samples. To align features with the same label based on the semantic feature in the graph semantic space, Graph Semantic Alignment (GSA) is performed to focus on the shape and the morphological differences between the lesions and their surrounding regions. RESULTS: Comprehensive experiments have been performed on two OCT image datasets. Compared to state-of-the-art methods, the proposed method can achieve better segmentation. CONCLUSION: The proposed fundus lesion segmentation method can be trained with labeled OCT images from multiple manufacturers' scanners and be tested on an unseen manufacturer's scanner with better domain generalization. SIGNIFICANCE: The proposed method can be used in routine clinical occasions when an unseen manufacturer's OCT image is available for a patient.

Cross-Modal Consistency for Single-Modal MR Image Segmentation.

OBJECTIVE: Multi-modal magnetic resonance (MR) image segmentation is an important task in disease diagnosis and treatment, but it is usually difficult to obtain multiple modalities for a single patient in clinical applications. To address these issues, a cross-modal consistency framework is proposed for a single-modal MR image segmentation. METHODS: To enable single-modal MR image segmentation in the inference stage, a weighted cross-entropy loss and a pixel-level feature consistency loss are proposed to train the target network with the guidance of the teacher network and the auxiliary network. To fuse dual-modal MR images in the training stage, the cross-modal consistency is measured according to Dice similarity entropy loss and Dice similarity contrastive loss, so as to maximize the prediction similarity of the teacher network and the auxiliary network. To reduce the difference in image contrast between different MR images for the same organs, a contrast alignment network is proposed to align input images with different contrasts to reference images with a good contrast. RESULTS: Comprehensive experiments have been performed on a publicly available prostate dataset and an in-house pancreas dataset to verify the effectiveness of the proposed method. Compared to state-of-the-art methods, the proposed method can achieve better segmentation. CONCLUSION: The proposed image segmentation method can fuse dual-modal MR images in the training stage and only need one-modal MR images in the inference stage. SIGNIFICANCE: The proposed method can be used in routine clinical occasions when only single-modal MR image with variable contrast is available for a patient.

Two-stage adversarial learning based unsupervised domain adaptation for retinal OCT segmentation.

BACKGROUND: Deep learning based optical coherence tomography (OCT) segmentation methods have achieved excellent results, allowing quantitative analysis of large-scale data. However, OCT images are often acquired by different devices or under different imaging protocols, which leads to serious domain shift problem. This in turn results in performance degradation of segmentation models. PURPOSE: Aiming at the domain shift problem, we propose a two-stage adversarial learning based network (TSANet) that accomplishes unsupervised cross-domain OCT segmentation. METHODS: In the first stage, a Fourier transform based approach is adopted to reduce image style differences from the image level. Then, adversarial learning networks, including a segmenter and a discriminator, are designed to achieve inter-domain consistency in the segmentation output. In the second stage, pseudo labels of selected unlabeled target domain training data are used to fine-tune the segmenter, which further improves its generalization capability. The proposed method was tested on cross-domain datasets for choroid or retinoschisis segmentation tasks. For choroid segmentation, the model was trained on 400 images and validated on 100 images from the source domain, and then trained on 1320 unlabeled images and tested on 330 images from target domain I, and also trained on 400 unlabeled images and tested on 200 images from target domain II. For retinoschisis segmentation, the model was trained on 1284 images and validated on 312 images from the source domain, and then trained on 1024 unlabeled images and tested on 200 images from the target domain. RESULTS: The proposed method achieved significantly improved results over that without domain adaptation, with improvement of 8.34%, 55.82% and 3.53% in intersection over union (IoU) respectively for the three test sets. The performance is better than some state-of-the-art domain adaptation methods. CONCLUSIONS: The proposed TSANet, with image level adaptation, feature level adaptation and pseudo-label based fine-tuning, achieved excellent cross-domain generalization. This alleviates the burden of obtaining additional manual labels when adapting the deep learning model to new OCT data.

DRFNet: a deep radiomic fusion network for nAMD/PCV differentiation in OCT images.

Objective.Neovascular age-related macular degeneration (nAMD) and polypoidal choroidal vasculopathy (PCV) present many similar clinical features. However, there are significant differences in the progression of nAMD and PCV. and it is crucial to make accurate diagnosis for treatment. In this paper, we propose a structure-radiomic fusion network (DRFNet) to differentiate PCV and nAMD in optical coherence tomography (OCT) images.Approach.The subnetwork (RIMNet) is designed to automatically segment the lesion of nAMD and PCV. Another subnetwork (StrEncoder) is designed to extract deep structural features of the segmented lesion. The subnetwork (RadEncoder) is designed to extract radiomic features from the segmented lesions based on radiomics. 305 eyes (155 with nAMD and 150 with PCV) are included and manually annotated CNV region in this study. The proposed method was trained and evaluated by 4-fold cross validation using the collected data and was compared with the advanced differentiation methods.Main results.The proposed method achieved high classification performace of nAMD/PCV differentiation in OCT images, which was an improvement of 4.68 compared with other best method.Significance. The presented structure-radiomic fusion network (DRFNet) has great performance of diagnosing nAMD and PCV and high clinical value by using OCT instead of indocyanine green angiography.

Automatic zoning for retinopathy of prematurity with a key area location system.

Retinopathy of prematurity (ROP) usually occurs in premature or low birth weight infants and has been an important cause of childhood blindness worldwide. Diagnosis and treatment of ROP are mainly based on stage, zone and disease, where the zone is more important than the stage for serious ROP. However, due to the great subjectivity and difference of ophthalmologists in the diagnosis of ROP zoning, it is challenging to achieve accurate and objective ROP zoning diagnosis. To address it, we propose a new key area location (KAL) system to achieve automatic and objective ROP zoning based on its definition, which consists of a key point location network and an object detection network. Firstly, to achieve the balance between real-time and high-accuracy, a lightweight residual heatmap network (LRH-Net) is designed to achieve the location of the optic disc (OD) and macular center, which transforms the location problem into a pixel-level regression problem based on the heatmap regression method and maximum likelihood estimation theory. In addition, to meet the needs of clinical accuracy and real-time detection, we use the one-stage object detection framework Yolov3 to achieve ROP lesion location. Finally, the experimental results have demonstrated that the proposed KAL system has achieved better performance on key point location (6.13 and 17.03 pixels error for OD and macular center location) and ROP lesion location (93.05% for AP50), and the ROP zoning results based on it have good consistency with the results manually labeled by clinicians, which can support clinical decision-making and help ophthalmologists correctly interpret ROP zoning, reducing subjective differences of diagnosis and increasing the interpretability of zoning results.

Evaluation of glymphatic system activity by diffusion tensor image analysis along the perivascular space in presbycusis.

PURPOSE: Previous studies have suggested that presbycusis (age-related hearing loss) is accompanied with cognitive decline and dementia. However, the neural mechanism underlying the cognitive decline in presbycusis remains unclear. This study aimed to evaluate the glymphatic system function in presbycusis patients compared to healthy controls using diffusion tensor imaging (DTI) with the perivascular space (DTI-ALPS) method. METHODS: DTI scans were obtained from 30 presbycusis patients with cognitive decline (PCD), 30 presbycusis patients with no cognitive decline (PNCD) and 40 age-, gender-, and education-matched healthy controls (HCs). The DTI-ALPS index was calculated for each group. We evaluated the differences in the DTI-ALPS index among PCD, PNCD and HCs. In addition, we conducted a correlation analysis between the DTI-ALPS index and cognitive performance. RESULTS: There were significant differences of the DTI-ALPS index among three groups. Post-hoc analysis suggested that the DTI-ALPS index in PCD was significantly lower patients in relative to PNCD and HCs (1.49147 vs. 1.57441 vs. 1.62020, p < 0.001). After correcting for age, gender, and education, the DTI-ALPS index is positively correlated with the MoCA scores (rho = 0.426, p = 0.026). CONCLUSION: Presbycusis patients with cognitive impairment exhibited decreased glymphatic activity than those without cognitive impairment and HCs. The DTI-ALPS index may provide useful disease progression or treatment biomarkers for patients with presbycusis as an indicator of modulation of glymphatic activity.

A Multi-Scale Fusion and Transformer Based Registration Guided Speckle Noise Reduction for OCT Images.

Optical coherence tomography (OCT) images are inevitably affected by speckle noise because OCT is based on low-coherence interference. Multi-frame averaging is one of the effective methods to reduce speckle noise. Before averaging, the misalignment between images must be calibrated. In this paper, in order to reduce misalignment between images caused during the acquisition, a novel multi-scale fusion and Transformer based (MsFTMorph) method is proposed for deformable retinal OCT image registration. The proposed method captures global connectivity and locality with convolutional vision transformer and also incorporates a multi-resolution fusion strategy for learning the global affine transformation. Comparative experiments with other state-of-the-art registration methods demonstrate that the proposed method achieves higher registration accuracy. Guided by the registration, subsequent multi-frame averaging shows better results in speckle noise reduction. The noise is suppressed while the edges can be preserved. In addition, our proposed method has strong cross-domain generalization, which can be directly applied to images acquired by different scanners with different modes.

Development and evaluation of multimodal AI for diagnosis and triage of ophthalmic diseases using ChatGPT and anterior segment images: protocol for a two-stage cross-sectional study.

Introduction: Artificial intelligence (AI) technology has made rapid progress for disease diagnosis and triage. In the field of ophthalmic diseases, image-based diagnosis has achieved high accuracy but still encounters limitations due to the lack of medical history. The emergence of ChatGPT enables human-computer interaction, allowing for the development of a multimodal AI system that integrates interactive text and image information. Objective: To develop a multimodal AI system using ChatGPT and anterior segment images for diagnosing and triaging ophthalmic diseases. To assess the AI system's performance through a two-stage cross-sectional study, starting with silent evaluation and followed by early clinical evaluation in outpatient clinics. Methods and analysis: Our study will be conducted across three distinct centers in Shanghai, Nanjing, and Suqian. The development of the smartphone-based multimodal AI system will take place in Shanghai with the goal of achieving ≥90% sensitivity and ≥95% specificity for diagnosing and triaging ophthalmic diseases. The first stage of the cross-sectional study will explore the system's performance in Shanghai's outpatient clinics. Medical histories will be collected without patient interaction, and anterior segment images will be captured using slit lamp equipment. This stage aims for ≥85% sensitivity and ≥95% specificity with a sample size of 100 patients. The second stage will take place at three locations, with Shanghai serving as the internal validation dataset, and Nanjing and Suqian as the external validation dataset. Medical history will be collected through patient interviews, and anterior segment images will be captured via smartphone devices. An expert panel will establish reference standards and assess AI accuracy for diagnosis and triage throughout all stages. A one-vs.-rest strategy will be used for data analysis, and a post-hoc power calculation will be performed to evaluate the impact of disease types on AI performance. Discussion: Our study may provide a user-friendly smartphone-based multimodal AI system for diagnosis and triage of ophthalmic diseases. This innovative system may support early detection of ocular abnormalities, facilitate establishment of a tiered healthcare system, and reduce the burdens on tertiary facilities. Trial registration: The study was registered in ClinicalTrials.gov on June 25th, 2023 (NCT05930444).

Uncertainty-inspired open set learning for retinal anomaly identification.

Failure to recognize samples from the classes unseen during training is a major limitation of artificial intelligence in the real-world implementation for recognition and classification of retinal anomalies. We establish an uncertainty-inspired open set (UIOS) model, which is trained with fundus images of 9 retinal conditions. Besides assessing the probability of each category, UIOS also calculates an uncertainty score to express its confidence. Our UIOS model with thresholding strategy achieves an F1 score of 99.55%, 97.01% and 91.91% for the internal testing set, external target categories (TC)-JSIEC dataset and TC-unseen testing set, respectively, compared to the F1 score of 92.20%, 80.69% and 64.74% by the standard AI model. Furthermore, UIOS correctly predicts high uncertainty scores, which would prompt the need for a manual check in the datasets of non-target categories retinal diseases, low-quality fundus images, and non-fundus images. UIOS provides a robust method for real-world screening of retinal anomalies.

MOCSS: Multi-omics data clustering and cancer subtyping via shared and specific representation learning.

Cancer is an extremely complex disease and each type of cancer usually has several different subtypes. Multi-omics data can provide more comprehensive biological information for identifying and discovering cancer subtypes. However, existing unsupervised cancer subtyping methods cannot effectively learn comprehensive shared and specific information of multi-omics data. Therefore, a novel method is proposed based on shared and specific representation learning. For each omics data, two autoencoders are applied to extract shared and specific information, respectively. To reduce redundancy and mutual interference, orthogonality constraint is introduced to separate shared and specific information. In addition, contrastive learning is applied to align the shared information and strengthen their consistency. Finally, the obtained shared and specific information for all samples are used for clustering tasks to achieve cancer subtyping. Experimental results demonstrate that the proposed method can effectively capture shared and specific information of multi-omics data and outperform other state-of-the-art methods on cancer subtyping.

A deep learning-based automatic tool for measuring the lengths of linear scars: forensic applications.

It is important to measure scars in forensic and clinical medicine. In practice, scars are mostly manually measured, and the results are diverse and influenced by various subjective factors. With the development of digital image technology and artificial intelligence, noncontact and automatic photogrammetry has been gradually used in some practical applications. In this article, we propose an automatic method for measuring the length of linear scars based on multiview stereo and deep learning, which combines the 3D reconstruction algorithm of structure from motion and the image segmentation algorithm based on a convolutional neural network. With a few pictures taken by a smart phone, automatic segmentation and measurement of scars can be realized. The reliability of the measurement was first demonstrated through simulation experiments on five artificial scars, giving errors of length <5%. Then, experiment results on 30 clinical scar samples showed that our measurements were in high agreement with manual measurements, with an average error of 3.69%. Our study demonstrates that the application of photogrammetry in scar measurement is effective and that the deep learning technique can realize the automation of scar measurement with high accuracy.

Accurate C/D ratio estimation with elliptical fitting for OCT image based on joint segmentation and detection network.

Proper estimation of the cup-to-disc ratio (C/D ratio) plays a significant role in ophthalmic examinations, and it is urgent to improve the efficiency of C/D ratio automatic measurement. Therefore, we propose a new method for measuring the C/D ratio of OCTs in normal subjects. Firstly, the end-to-end deep convolution network is used to segment and detect the inner limiting membrane (ILM) and the two Bruch's membrane opening (BMO) terminations. Then, we introduce an ellipse fitting technique to post-process the edge of the optic disc. Finally, the proposed method is evaluated on 41 normal subjects using the optic-disc-area scanning mode of three machines: BV1000, Topcon 3D OCT-1, and Nidek ARK-1. In addition, pairwise correlation analyses are carried out to compare the C/D ratio measurement method of BV1000 to existing commercial OCT machines as well as other state-of-the-art methods. The correlation coefficient between the C/D ratio calculated by BV1000 and the C/D ratio calculated by manual annotation is 0.84, which indicates that the proposed method has a strong correlation with the results of manual annotation by ophthalmologists. Moreover, in comparison between BV1000, Topcon and Nidek in practical screening among normal subjects, the proportion of the C/D ratio less than 0.6 calculated by BV1000 accounts for 96.34%, which is the closest to the clinical statistics among the three OCT machines. The above experimental results and analysis show that the proposed method performs well in cup and disc detection and C/D ratio measurement, and compared with the existing commercial OCT equipment, the C/D ratio measurement results are relatively close to reality, which has certain clinical application value.

Segmentation of retinal detachment and retinoschisis in OCT images based on complementary multi-class segmentation networks.

Retinal detachment (RD) and retinoschisis (RS) are the main complications leading to vision loss in high myopia. Accurate segmentation of RD and RS, including its subcategories (outer, middle, and inner retinoschisis) in optical coherence tomography images is of great clinical significance in the diagnosis and management of high myopia. For this multi-class segmentation task, we propose a novel framework named complementary multi-class segmentation networks. Based on domain knowledge, a three-class segmentation path (TSP) and a five-class segmentation path (FSP) are designed, and their outputs are integrated through additional decision fusion layers to achieve improved segmentation in a complementary manner. In TSP, a cross-fusion global feature module is adopted to achieve global receptive field. In FSP, a novel three-dimensional contextual information perception module is proposed to capture long-range contexts, and a classification branch is designed to provide useful features for segmentation. A new category loss is also proposed in FSP to help better identify the lesion categories. Experiment results show that the proposed method achieves superior performance for joint segmentation of RD and the three subcategories of RS, with an average Dice coefficient of 84.83%.

Interactive Skin Wound Segmentation Based on Feature Augment Networks.

Skin wound segmentation in photographs allows non-invasive analysis of wounds that supports dermatological diagnosis and treatment. In this paper, we propose a novel feature augment network (FANet) to achieve automatic segmentation of skin wounds, and design an interactive feature augment network (IFANet) to provide interactive adjustment on the automatic segmentation results. The FANet contains the edge feature augment (EFA) module and the spatial relationship feature augment (SFA) module, which can make full use of the notable edge information and the spatial relationship information be-tween the wound and the skin. The IFANet, with FANet as the backbone, takes the user interactions and the initial result as inputs, and outputs the refined segmentation result. The pro-posed networks were tested on a dataset composed of miscellaneous skin wound images, and a public foot ulcer segmentation challenge dataset. The results indicate that the FANet gives good segmentation results while the IFANet can effectively improve them based on simple marking. Comprehensive comparative experiments show that our proposed networks outperform some other existing automatic or interactive segmentation methods, respectively.

Multi-scale and local feature guidance network for corneal nerve fiber segmentation.

Objective. Corneal confocal microscopy (CCM) is a rapid and non-invasive ophthalmic imaging technique that can reveal corneal nerve fiber. The automatic segmentation of corneal nerve fiber in CCM images is vital for the subsequent abnormality analysis, which is the main basis for the early diagnosis of degenerative neurological systemic diseases such as diabetic peripheral neuropathy.Approach. In this paper, a U-shape encoder-decoder structure based multi-scale and local feature guidance neural network (MLFGNet) is proposed for the automatic corneal nerve fiber segmentation in CCM images. Three novel modules including multi-scale progressive guidance (MFPG) module, local feature guided attention (LFGA) module, and multi-scale deep supervision (MDS) module are proposed and applied in skip connection, bottom of the encoder and decoder path respectively, which are designed from both multi-scale information fusion and local information extraction perspectives to enhance the network's ability to discriminate the global and local structure of nerve fibers. The proposed MFPG module solves the imbalance between semantic information and spatial information, the LFGA module enables the network to capture attention relationships on local feature maps and the MDS module fully utilizes the relationship between high-level and low-level features for feature reconstruction in the decoder path.Main results. The proposed MLFGNet is evaluated on three CCM image Datasets, the Dice coefficients reach 89.33%, 89.41%, and 88.29% respectively.Significance. The proposed method has excellent segmentation performance for corneal nerve fibers and outperforms other state-of-the-art methods.

Self-Guided Optimization Semi-Supervised Method for Joint Segmentation of Macular Hole and Cystoid Macular Edema in Retinal OCT Images.

Macular hole (MH) and cystoid macular edema (CME) are two common retinal pathologies that cause vision loss. Accurate segmentation of MH and CME in retinal OCT images can greatly aid ophthalmologists to evaluate the relevant diseases. However, it is still challenging as the complicated pathological features of MH and CME in retinal OCT images, such as the diversity of morphologies, low imaging contrast, and blurred boundaries. In addition, the lack of pixel-level annotation data is one of the important factors that hinders the further improvement of segmentation accuracy. Focusing on these challenges, we propose a novel self-guided optimization semi-supervised method termed Semi-SGO for joint segmentation of MH and CME in retinal OCT images. Aiming to improve the model's ability to learn the complicated pathological features of MH and CME, while alleviating the feature learning tendency problem that may be caused by the introduction of skip-connection in U-shaped segmentation architecture, we develop a novel dual decoder dual-task fully convolutional neural network (D3T-FCN). Meanwhile, based on our proposed D3T-FCN, we introduce a knowledge distillation technique to further design a novel semi-supervised segmentation method called Semi-SGO, which can leverage unlabeled data to further improve the segmentation accuracy. Comprehensive experimental results show that our proposed Semi-SGO outperforms other state-of-the-art segmentation networks. Furthermore, we also develop an automatic method for measuring the clinical indicators of MH and CME to validate the clinical significance of our proposed Semi-SGO. The code will be released on Github 1,2.

Graph Attention U-Net for Retinal Layer Surface Detection and Choroid Neovascularization Segmentation in OCT Images.

Choroidal neovascularization (CNV) is a typical symptom of age-related macular degeneration (AMD) and is one of the leading causes for blindness. Accurate segmentation of CNV and detection of retinal layers are critical for eye disease diagnosis and monitoring. In this paper, we propose a novel graph attention U-Net (GA-UNet) for retinal layer surface detection and CNV segmentation in optical coherence tomography (OCT) images. Due to retinal layer deformation caused by CNV, it is challenging for existing models to segment CNV and detect retinal layer surfaces with the correct topological order. We propose two novel modules to address the challenge. The first module is a graph attention encoder (GAE) in a U-Net model that automatically integrates topological and pathological knowledge of retinal layers into the U-Net structure to achieve effective feature embedding. The second module is a graph decorrelation module (GDM) that takes reconstructed features by the decoder of the U-Net as inputs, it then decorrelates and removes information unrelated to retinal layer for improved retinal layer surface detection. In addition, we propose a new loss function to maintain the correct topological order of retinal layers and the continuity of their boundaries. The proposed model learns graph attention maps automatically during training and performs retinal layer surface detection and CNV segmentation simultaneously with the attention maps during inference. We evaluated the proposed model on our private AMD dataset and another public dataset. Experiment results show that the proposed model outperformed the competing methods for retinal layer surface detection and CNV segmentation and achieved new state of the arts on the datasets.

LKG-Net: lightweight keratoconus grading network based on corneal topography.

Keratoconus (KC) is a noninflammatory ectatic disease characterized by progressive thinning and an apical cone-shaped protrusion of the cornea. In recent years, more and more researchers have been committed to automatic and semi-automatic KC detection based on corneal topography. However, there are few studies about the severity grading of KC, which is particularly important for the treatment of KC. In this work, we propose a lightweight KC grading network (LKG-Net) for 4-level KC grading (Normal, Mild, Moderate, and Severe). First of all, we use depth-wise separable convolution to design a novel feature extraction block based on the self-attention mechanism, which can not only extract rich features but also reduce feature redundancy and greatly reduce the number of parameters. Then, to improve the model performance, a multi-level feature fusion module is proposed to fuse features from the upper and lower levels to obtain more abundant and effective features. The proposed LKG-Net was evaluated on the corneal topography of 488 eyes from 281 people with 4-fold cross-validation. Compared with other state-of-the-art classification methods, the proposed method achieves 89.55% for weighted recall (W_R), 89.98% for weighted precision (W_P), 89.50% for weighted F1 score (W_F1) and 94.38% for Kappa, respectively. In addition, the LKG-Net is also evaluated on KC screening, and the experimental results show the effectiveness.

OCT2Former: A retinal OCT-angiography vessel segmentation transformer.

BACKGROUND AND OBJECTIVE: Retinal vessel segmentation plays an important role in the automatic retinal disease screening and diagnosis. How to segment thin vessels and maintain the connectivity of vessels are the key challenges of the retinal vessel segmentation task. Optical coherence tomography angiography (OCTA) is a noninvasive imaging technique that can reveal high-resolution retinal vessels. Aiming at make full use of its characteristic of high resolution, a new end-to-end transformer based network named as OCT2Former (OCT-a Transformer) is proposed to segment retinal vessel accurately in OCTA images. METHODS: The proposed OCT2Former is based on encoder-decoder structure, which mainly includes dynamic transformer encoder and lightweight decoder. Dynamic transformer encoder consists of dynamic token aggregation transformer and auxiliary convolution branch, in which the multi-head dynamic token aggregation attention based dynamic token aggregation transformer is designed to capture the global retinal vessel context information from the first layer throughout the network and the auxiliary convolution branch is proposed to compensate for the lack of inductive bias of the transformer and assist in the efficient feature extraction. A convolution based lightweight decoder is proposed to decode features efficiently and reduce the complexity of the proposed OCT2Former. RESULTS: The proposed OCT2Former is validated on three publicly available datasets i.e. OCTA-SS, ROSE-1, OCTA-500 (subset OCTA-6M and OCTA-3M). The Jaccard indexes of the proposed OCT2Former on these datasets are 0.8344, 0.7855, 0.8099 and 0.8513, respectively, outperforming the best convolution based network 1.43, 1.32, 0.75 and 1.46%, respectively. CONCLUSION: The experimental results have demonstrated that the proposed OCT2Former can achieve competitive performance on retinal OCTA vessel segmentation tasks.