Applications of natural language processing tools in the surgical journey.

Background: Natural language processing tools are becoming increasingly adopted in multiple industries worldwide. They have shown promising results however their use in the field of surgery is under-recognised. Many trials have assessed these benefits in small settings with promising results before large scale adoption can be considered in surgery. This study aims to review the current research and insights into the potential for implementation of natural language processing tools into surgery. Methods: A narrative review was conducted following a computer-assisted literature search on Medline, EMBASE and Google Scholar databases. Papers related to natural language processing tools and consideration into their use for surgery were considered. Results: Current applications of natural language processing tools within surgery are limited. From the literature, there is evidence of potential improvement in surgical capability and service delivery, such as through the use of these technologies to streamline processes including surgical triaging, data collection and auditing, surgical communication and documentation. Additionally, there is potential to extend these capabilities to surgical academia to improve processes in surgical research and allow innovation in the development of educational resources. Despite these outcomes, the evidence to support these findings are challenged by small sample sizes with limited applicability to broader settings. Conclusion: With the increasing adoption of natural language processing technology, such as in popular forms like ChatGPT, there has been increasing research in the use of these tools within surgery to improve surgical workflow and efficiency. This review highlights multifaceted applications of natural language processing within surgery, albeit with clear limitations due to the infancy of the infrastructure available to leverage these technologies. There remains room for more rigorous research into broader capability of natural language processing technology within the field of surgery and the need for cross-sectoral collaboration to understand the ways in which these algorithms can best be integrated.

Artificial intelligence to automate assessment of ocular and periocular measurements.

PURPOSE: To develop and validate a deep learning facial landmark detection network to automate the assessment of periocular anthropometric measurements. METHODS: Patients presenting to the ophthalmology clinic were prospectively enrolled and had their images taken using a standardised protocol. Facial landmarks were segmented on the images to enable calculation of marginal reflex distance (MRD) 1 and 2, palpebral fissure height (PFH), inner intercanthal distance (IICD), outer intercanthal distance (OICD), interpupillary distance (IPD) and horizontal palpebral aperture (HPA). These manual segmentations were used to train a machine learning algorithm to automatically detect facial landmarks and calculate these measurements. The main outcomes were the mean absolute error and intraclass correlation coefficient. RESULTS: A total of 958 eyes from 479 participants were included. The testing set consisted of 290 eyes from 145 patients. The AI algorithm demonstrated close agreement with human measurements, with mean absolute errors ranging from 0.22 mm for IPD to 0.88 mm for IICD. The intraclass correlation coefficients indicated excellent reliability (ICC > 0.90) for MRD1, MRD2, PFH, OICD, IICD, and IPD, while HPA showed good reliability (ICC 0.84). The landmark detection model was highly accurate and achieved a mean error rate of 0.51% and failure rate at 0.1 of 0%. CONCLUSION: The automated facial landmark detection network provided accurate and reliable periocular measurements. This may help increase the objectivity of periocular measurements in the clinic and may facilitate remote assessment of patients with tele-health.

Targeting macrophages with multifunctional nanoparticles to detect and prevent atherosclerotic cardiovascular disease.

Despite the emergence of novel diagnostic, pharmacological, interventional and prevention strategies, atherosclerotic cardiovascular disease remains a significant cause of morbidity and mortality. Nanoparticle-based platforms encompass diverse imaging, delivery and pharmacological properties that provide novel opportunities for refining diagnostic and therapeutic interventions for atherosclerosis at the cellular and molecular level. Macrophages play a critical role in atherosclerosis and therefore represent an important disease-related diagnostic and therapeutic target, especially given their inherent ability for passive and active nanoparticle uptake. In this review, we discuss an array of inorganic, carbon-based and lipid-based nanoparticles that provide magnetic, radiographic and fluorescent imaging capabilities for a range of highly promising research and clinical applications in atherosclerosis. We discuss the design of nanoparticles that target a range of macrophage-related functions such as lipoprotein oxidation, cholesterol efflux, vascular inflammation and defective efferocytosis. We also provide examples of nanoparticle systems that were developed for other pathologies such as cancer and highlight their potential for repurposing in cardiovascular disease. Finally, we discuss the current state of play and the future of theranostic nanoparticles. Whilst this is not without its challenges, the array of multifunctional capabilities that are possible in nanoparticle design ensures they will be part of the next frontier of exciting new therapies that simultaneously improve the accuracy of plaque diagnosis and more effectively reduce atherosclerosis with limited side effects.

Coronary Atherosclerotic Plaque Activity and Risk of Myocardial Infarction.

BACKGROUND: Total coronary atherosclerotic plaque activity across the entire coronary arterial tree is associated with patient-level clinical outcomes. OBJECTIVES: We aimed to investigate whether vessel-level coronary atherosclerotic plaque activity is associated with vessel-level myocardial infarction. METHODS: In this secondary analysis of an international multicenter study of patients with recent myocardial infarction and multivessel coronary artery disease, we assessed vessel-level coronary atherosclerotic plaque activity using coronary 18F-sodium fluoride positron emission tomography to identify vessel-level myocardial infarction. RESULTS: Increased 18F-sodium fluoride uptake was found in 679 of 2,094 coronary arteries and 414 of 691 patients. Myocardial infarction occurred in 24 (4%) vessels with increased coronary atherosclerotic plaque activity and in 25 (2%) vessels without increased coronary atherosclerotic plaque activity (HR: 2.08; 95% CI: 1.16-3.72; P = 0.013). This association was not demonstrable in those treated with coronary revascularization (HR: 1.02; 95% CI: 0.47-2.25) but was notable in untreated vessels (HR: 3.86; 95% CI: 1.63-9.10; Pinteraction = 0.024). Increased coronary atherosclerotic plaque activity in multiple coronary arteries was associated with heightened patient-level risk of cardiac death or myocardial infarction (HR: 2.43; 95% CI: 1.37-4.30; P = 0.002) as well as first (HR: 2.19; 95% CI: 1.18-4.06; P = 0.013) and total (HR: 2.50; 95% CI: 1.42-4.39; P = 0.002) myocardial infarctions. CONCLUSIONS: In patients with recent myocardial infarction and multivessel coronary artery disease, coronary atherosclerotic plaque activity prognosticates individual coronary arteries and patients at risk for myocardial infarction.

Facial and mandibular landmark tracking with habitual head posture estimation using linear and fiducial markers.

This study compared the accuracy of facial landmark measurements using deep learning-based fiducial marker (FM) and arbitrary width reference (AWR) approaches. It quantitatively analysed mandibular hard and soft tissue lateral excursions and head tilting from consumer camera footage of 37 participants. A custom deep learning system recognised facial landmarks for measuring head tilt and mandibular lateral excursions. Circular fiducial markers (FM) and inter-zygion measurements (AWR) were validated against physical measurements using electrognathography and electronic rulers. Results showed notable differences in lower and mid-face estimations for both FM and AWR compared to physical measurements. The study also demonstrated the comparability of both approaches in assessing lateral movement, though fiducial markers exhibited variability in mid-face and lower face parameter assessments. Regardless of the technique applied, hard tissue movement was typically seen to be 30% less than soft tissue among the participants. Additionally, a significant number of participants consistently displayed a 5 to 10° head tilt.

Deep learning to predict lymph node status on pre-operative staging CT in patients with colon cancer.

INTRODUCTION: Lymph node (LN) metastases are an important determinant of survival in patients with colon cancer, but remain difficult to accurately diagnose on preoperative imaging. This study aimed to develop and evaluate a deep learning model to predict LN status on preoperative staging CT. METHODS: In this ambispective diagnostic study, a deep learning model using a ResNet-50 framework was developed to predict LN status based on preoperative staging CT. Patients with a preoperative staging abdominopelvic CT who underwent surgical resection for colon cancer were enrolled. Data were retrospectively collected from February 2007 to October 2019 and randomly separated into training, validation, and testing cohort 1. To prospectively test the deep learning model, data for testing cohort 2 was collected from October 2019 to July 2021. Diagnostic performance measures were assessed by the AUROC. RESULTS: A total of 1,201 patients (median [range] age, 72 [28-98 years]; 653 [54.4%] male) fulfilled the eligibility criteria and were included in the training (n = 401), validation (n = 100), testing cohort 1 (n = 500) and testing cohort 2 (n = 200). The deep learning model achieved an AUROC of 0.619 (95% CI 0.507-0.731) in the validation cohort. In testing cohort 1 and testing cohort 2, the AUROC was 0.542 (95% CI 0.489-0.595) and 0.486 (95% CI 0.403-0.568), respectively. CONCLUSION: A deep learning model based on a ResNet-50 framework does not predict LN status on preoperative staging CT in patients with colon cancer.

ReFs: A hybrid pre-training paradigm for 3D medical image segmentation.

Self-supervised learning (SSL) has achieved remarkable progress in medical image segmentation. The application of an SSL algorithm often follows a two-stage training process: using unlabeled data to perform label-free representation learning and fine-tuning the pre-trained model on the downstream tasks. One issue of this paradigm is that the SSL step is unaware of the downstream task, which may lead to sub-optimal feature representation for a target task. In this paper, we propose a hybrid pre-training paradigm that is driven by both self-supervised and supervised objectives. To achieve this, a supervised reference task is involved in self-supervised learning, aiming to improve the representation quality. Specifically, we employ the off-the-shelf medical image segmentation task as reference, and encourage learning a representation that (1) incurs low prediction loss on both SSL and reference tasks and (2) leads to a similar gradient when updating the feature extractor from either task. In this way, the reference task pilots SSL in the direction beneficial for the downstream segmentation. To this end, we propose a simple but effective gradient matching method to optimize the model towards a consistent direction, thus improving the compatibility of both SSL and supervised reference tasks. We call this hybrid pre-training paradigm reference-guided self-supervised learning (ReFs), and perform it on a large-scale unlabeled dataset and an additional reference dataset. The experimental results demonstrate its effectiveness on seven downstream medical image segmentation benchmarks.

Self-supervised pseudo multi-class pre-training for unsupervised anomaly detection and segmentation in medical images.

Unsupervised anomaly detection (UAD) methods are trained with normal (or healthy) images only, but during testing, they are able to classify normal and abnormal (or disease) images. UAD is an important medical image analysis (MIA) method to be applied in disease screening problems because the training sets available for those problems usually contain only normal images. However, the exclusive reliance on normal images may result in the learning of ineffective low-dimensional image representations that are not sensitive enough to detect and segment unseen abnormal lesions of varying size, appearance, and shape. Pre-training UAD methods with self-supervised learning, based on computer vision techniques, can mitigate this challenge, but they are sub-optimal because they do not explore domain knowledge for designing the pretext tasks, and their contrastive learning losses do not try to cluster the normal training images, which may result in a sparse distribution of normal images that is ineffective for anomaly detection. In this paper, we propose a new self-supervised pre-training method for MIA UAD applications, named Pseudo Multi-class Strong Augmentation via Contrastive Learning (PMSACL). PMSACL consists of a novel optimisation method that contrasts a normal image class from multiple pseudo classes of synthesised abnormal images, with each class enforced to form a dense cluster in the feature space. In the experiments, we show that our PMSACL pre-training improves the accuracy of SOTA UAD methods on many MIA benchmarks using colonoscopy, fundus screening and Covid-19 Chest X-ray datasets.

Coronary Atherosclerotic Plaque Activity and Future Coronary Events.

Importance: Recurrent coronary events in patients with recent myocardial infarction remain a major clinical problem. Noninvasive measures of coronary atherosclerotic disease activity have the potential to identify individuals at greatest risk. Objective: To assess whether coronary atherosclerotic plaque activity as assessed by noninvasive imaging is associated with recurrent coronary events in patients with myocardial infarction. Design, Setting, and Participants: This prospective, longitudinal, international multicenter cohort study recruited participants aged 50 years or older with multivessel coronary artery disease and recent (within 21 days) myocardial infarction between September 2015 and February 2020, with a minimum 2 years' follow-up. Intervention: Coronary 18F-sodium fluoride positron emission tomography and coronary computed tomography angiography. Main Outcomes and Measures: Total coronary atherosclerotic plaque activity was assessed by 18F-sodium fluoride uptake. The primary end point was cardiac death or nonfatal myocardial infarction but was expanded during study conduct to include unscheduled coronary revascularization due to lower than anticipated primary event rates. Results: Among 2684 patients screened, 995 were eligible, 712 attended for imaging, and 704 completed an interpretable scan and comprised the study population. The mean (SD) age of participants was 63.8 (8.2) years, and most were male (601 [85%]). Total coronary atherosclerotic plaque activity was identified in 421 participants (60%). After a median follow-up of 4 years (IQR, 3-5 years), 141 participants (20%) experienced the primary end point: 9 had cardiac death, 49 had nonfatal myocardial infarction, and 83 had unscheduled coronary revascularizations. Increased coronary plaque activity was not associated with the primary end point (hazard ratio [HR], 1.25; 95% CI, 0.89-1.76; P = .20) or unscheduled revascularization (HR, 0.98; 95% CI, 0.64-1.49; P = .91) but was associated with the secondary end point of cardiac death or nonfatal myocardial infarction (47 of 421 patients with high plaque activity [11.2%] vs 19 of 283 with low plaque activity [6.7%]; HR, 1.82; 95% CI, 1.07-3.10; P = .03) and all-cause mortality (30 of 421 patients with high plaque activity [7.1%] vs 9 of 283 with low plaque activity [3.2%]; HR, 2.43; 95% CI, 1.15-5.12; P = .02). After adjustment for differences in baseline clinical characteristics, coronary angiography findings, and Global Registry of Acute Coronary Events score, high coronary plaque activity was associated with cardiac death or nonfatal myocardial infarction (HR, 1.76; 95% CI, 1.00-3.10; P = .05) but not with all-cause mortality (HR, 2.01; 95% CI, 0.90-4.49; P = .09). Conclusions and Relevance: In this cohort study of patients with recent myocardial infarction, coronary atherosclerotic plaque activity was not associated with the primary composite end point. The findings suggest that risk of cardiovascular death or myocardial infarction in patients with elevated plaque activity warrants further research to explore its incremental prognostic implications.

Normal periocular anthropometric measurements in an Australian population.

PURPOSE: To report the normative ocular and periocular anthropometric measurements in an Australian cohort and investigate how these may be affected age, gender, and ethnicity. METHODS: Prospective study of patients presenting to the Royal Adelaide Hospital. Patient with orbital or eyelid disease, previous surgery, craniofacial abnormalities, pupil abnormalities, strabismus, and poor image quality was excluded. Standardised photographs were taken in a well-illuminated room. A green dot with a diameter of 24 mm was placed on the participant's foreheads for calibration between pixels and millimetres. Ocular and periocular landmarks were segmented to calculate the periorbital measurements. Independent sample t test was used to compare male and female subjects, Pearson's correlation was used to correlate periocular dimensions with age, and ANOVA with Bonferroni was used to compare periocular dimension between ethnic groups. RESULTS: Seven hundred and sixty eyes from 380 participants (215 female, mean age 58 ± 18 years) were included. The mean marginal reflex distance (MRD) 1 was 3.5 mm and decreased with increasing age (r = - 0.09, p = 0.01) and MRD 2 was 5.2 mm. Compared to Caucasians, African subjects had a significantly larger interpupillary distance and outer intercanthal distance, whereas East Asians had a significantly larger inner intercanthal distance (p < 0.05). The values of marginal reflex distance 2, palpebral fissure height, horizontal palpebral aperture, inner intercanthal distance, interpupillary distance and outer intercanthal distance were significantly higher in male subjects than female subjects (p < 0.05). CONCLUSIONS: Normative periocular dimensions may vary according to age, gender, and ethnicity. An understanding of normal periocular dimensions is important in the evaluation of orbital disease across different ethnic groups and may serve as reference points for oculoplastic surgery and industry.

Baseline gut microbiota composition is associated with oral mucositis and tumour recurrence in patients with head and neck cancer: a pilot study.

PURPOSE: Mounting evidence suggests that the gut microbiome influences radiotherapy efficacy and toxicity by modulating immune signalling. However, its contribution to radiotherapy outcomes in head and neck cancer (HNC) is yet to be investigated. This study, therefore, aimed to uncover associations between an individual's pre-therapy gut microbiota and (i) severity of radiotherapy-induced oral mucositis (OM), and (ii) recurrence risk in patients with HNC. METHODS: In this prospective pilot study, 20 patients with HNC scheduled to receive radiotherapy or chemoradiotherapy were recruited. Stool samples were collected before treatment and microbial composition was analysed using 16S rRNA gene sequencing. OM severity was assessed using the NCI-CTCAE scoring system. Patients were also followed for 12 months of treatment completion to assess tumour recurrence. RESULTS: Overall, 80% of the patients were male with a median age of 65.5 years. Fifty-three percent experienced mild/moderate OM while 47% developed severe OM. Furthermore, 18% experienced tumour relapse within 1 year of treatment completion. A pre-treatment microbiota enriched of Eubacterium, Victivallis, and Ruminococcus was associated with severe OM. Conversely, a higher relative abundance of immunomodulatory microbes Faecalibacterium, Prevotella, and Phascolarctobacterium was associated with a lower risk of tumour recurrence. CONCLUSION: Our results indicate that a patient's gut microbiota composition at the start of treatment is linked to OM severity and recurrence risk. We now seek to validate these findings to determine their ability to predict treatment outcomes in HNC, with the goal of using this data to inform second-generation microbial therapeutics to optimise treatment outcomes for patients with HNC.

Development and external validation of automated detection, classification, and localization of ankle fractures: inside the black box of a convolutional neural network (CNN).

PURPOSE: Convolutional neural networks (CNNs) are increasingly being developed for automated fracture detection in orthopaedic trauma surgery. Studies to date, however, are limited to providing classification based on the entire image-and only produce heatmaps for approximate fracture localization instead of delineating exact fracture morphology. Therefore, we aimed to answer (1) what is the performance of a CNN that detects, classifies, localizes, and segments an ankle fracture, and (2) would this be externally valid? METHODS: The training set included 326 isolated fibula fractures and 423 non-fracture radiographs. The Detectron2 implementation of the Mask R-CNN was trained with labelled and annotated radiographs. The internal validation (or 'test set') and external validation sets consisted of 300 and 334 radiographs, respectively. Consensus agreement between three experienced fellowship-trained trauma surgeons was defined as the ground truth label. Diagnostic accuracy and area under the receiver operator characteristic curve (AUC) were used to assess classification performance. The Intersection over Union (IoU) was used to quantify accuracy of the segmentation predictions by the CNN, where a value of 0.5 is generally considered an adequate segmentation. RESULTS: The final CNN was able to classify fibula fractures according to four classes (Danis-Weber A, B, C and No Fracture) with AUC values ranging from 0.93 to 0.99. Diagnostic accuracy was 89% on the test set with average sensitivity of 89% and specificity of 96%. External validity was 89-90% accurate on a set of radiographs from a different hospital. Accuracies/AUCs observed were 100/0.99 for the 'No Fracture' class, 92/0.99 for 'Weber B', 88/0.93 for 'Weber C', and 76/0.97 for 'Weber A'. For the fracture bounding box prediction by the CNN, a mean IoU of 0.65 (SD ± 0.16) was observed. The fracture segmentation predictions by the CNN resulted in a mean IoU of 0.47 (SD ± 0.17). CONCLUSIONS: This study presents a look into the 'black box' of CNNs and represents the first automated delineation (segmentation) of fracture lines on (ankle) radiographs. The AUC values presented in this paper indicate good discriminatory capability of the CNN and substantiate further study of CNNs in detecting and classifying ankle fractures. LEVEL OF EVIDENCE: II, Diagnostic imaging study.

Mutual Information Neural Estimation for Unsupervised Multi-Modal Registration of Brain Images.

Many applications in image-guided surgery and therapy require fast and reliable non-linear, multi-modal image registration. Recently proposed unsupervised deep learning-based registration methods have demonstrated superior per-formance compared to iterative methods in just a fraction of the time. Most of the learning-based methods have focused on mono-modal image registration. The extension to multi-modal registration depends on the use of an appropriate similarity function, such as the mutual information (MI). We propose guiding the training of a deep learning-based registration method with MI estimation between an image-pair in an end-to-end trainable network. Our results show that a small, 2-layer network produces competitive results in both mono- and multi-modal registration, with sub-second run-times. Comparisons to both iterative and deep learning-based methods show that our MI-based method produces topologically and qualitatively superior results with an extremely low rate of non-diffeomorphic transformations. Real-time clinical application will benefit from a better visual matching of anatomical structures and less registration failures/outliers.

3D-Printed Micro Lens-in-Lens for In Vivo Multimodal Microendoscopy.

Multimodal microendoscopes enable co-located structural and molecular measurements in vivo, thus providing useful insights into the pathological changes associated with disease. However, different optical imaging modalities often have conflicting optical requirements for optimal lens design. For example, a high numerical aperture (NA) lens is needed to realize high-sensitivity fluorescence measurements. In contrast, optical coherence tomography (OCT) demands a low NA to achieve a large depth of focus. These competing requirements present a significant challenge in the design and fabrication of miniaturized imaging probes that are capable of supporting high-quality multiple modalities simultaneously. An optical design is demonstrated which uses two-photon 3D printing to create a miniaturized lens that is simultaneously optimized for these conflicting imaging modalities. The lens-in-lens design contains distinct but connected optical surfaces that separately address the needs of both fluorescence and OCT imaging within a lens of 330 µm diameter. This design shows an improvement in fluorescence sensitivity of >10x in contrast to more conventional fiber-optic design approaches. This lens-in-lens is then integrated into an intravascular catheter probe with a diameter of 520 µm. The first simultaneous intravascular OCT and fluorescence imaging of a mouse artery in vivo is reported.

CNN Attention Guidance for Improved Orthopedics Radiographic Fracture Classification.

Convolutional neural networks (CNNs) have gained significant popularity in orthopedic imaging in recent years due to their ability to solve fracture classification problems. A common criticism of CNNs is their opaque learning and reasoning process, making it difficult to trust machine diagnosis and the subsequent adoption of such algorithms in clinical setting. This is especially true when the CNN is trained with limited amount of medical data, which is a common issue as curating sufficiently large amount of annotated medical imaging data is a long and costly process. While interest has been devoted to explaining CNN learnt knowledge by visualizing network attention, the utilization of the visualized attention to improve network learning has been rarely investigated. This paper explores the effectiveness of regularizing CNN network with human-provided attention guidance on where in the image the network should look for answering clues. On two orthopedics radiographic fracture classification datasets, through extensive experiments we demonstrate that explicit human-guided attention indeed can direct correct network attention and consequently significantly improve classification performance. The development code for the proposed attention guidance is publicly available on https://github.com/zhibinliao89/fracture_attention_guidance.

Equivalent Carbon Number and Interclass Retention Time Conversion Enhance Lipid Identification in Untargeted Clinical Lipidomics.

Chromatography is often used as a method for reducing sample complexity prior to analysis by mass spectrometry, and the use of retention time (RT) is becoming increasingly popular to add valuable supporting information in lipid identification. The RT of lipids with the same headgroup in reversed-phase separation can be predicted using the equivalent carbon number (ECN) model. This model describes the effects of acyl chain length and degree of saturation on lipid RT. For the first time, we have found a robust correlation in the chromatographic separation of lipids with different headgroups that share the same fatty acid motive. This relationship can be exploited to perform interclass RT conversion (IC-RTC) by building a model from RT measurements from lipid standards that allows the prediction of RT of one lipid subclass based on another. Here, we utilize ECN modeling and IC-RTC to build a glycerophospholipid RT library with 517 entries based on 136 tandem mass spectrometry-characterized lipid RTs from NIST SRM-1950 plasma and lipid standards. The library was tested on a patient cohort undergoing coronary artery bypass grafting surgery (n = 37). A total of 156 unique circulating glycerophospholipids were identified, of which 52 (1 LPG, 24 PE, 5 PG, 18 PI, and 9 PS) were detected with IC-RTC, thereby demonstrating the utility of this technique for the identification of lipid species not found in commercial standards.

Multimodality Intravascular Imaging of High-Risk Coronary Plaque.

The majority of coronary atherothrombotic events presenting as myocardial infarction (MI) occur as a result of plaque rupture or erosion. Understanding the evolution from a stable plaque into a life-threatening, high-risk plaque is required for advancing clinical approaches to predict atherothrombotic events, and better treat coronary atherosclerosis. Unfortunately, none of the coronary imaging approaches used in clinical practice can reliably predict which plaques will cause an MI. Currently used imaging techniques mostly identify morphological features of plaques, but are not capable of detecting essential molecular characteristics known to be important drivers of future risk. To address this challenge, engineers, scientists, and clinicians have been working hand-in-hand to advance a variety of multimodality intravascular imaging techniques, whereby 2 or more complementary modalities are integrated into the same imaging catheter. Some of these have already been tested in early clinical studies, with other next-generation techniques also in development. This review examines these emerging hybrid intracoronary imaging techniques and discusses their strengths, limitations, and potential for clinical translation from both an engineering and clinical perspective.

Intra- and Inter-Pair Consistency for Semi-Supervised Gland Segmentation.

Accurate gland segmentation in histology tissue images is a critical but challenging task. Although deep models have demonstrated superior performance in medical image segmentation, they commonly require a large amount of annotated data, which are hard to obtain due to the extensive labor costs and expertise required. In this paper, we propose an intra- and inter-pair consistency-based semi-supervised (I2CS) model that can be trained on both labeled and unlabeled histology images for gland segmentation. Considering that each image contains glands and hence different images could potentially share consistent semantics in the feature space, we introduce a novel intra- and inter-pair consistency module to explore such consistency for learning with unlabeled data. It first characterizes the pixel-level relation between a pair of images in the feature space to create an attention map that highlights the regions with the same semantics but on different images. Then, it imposes a consistency constraint on the attention maps obtained from multiple image pairs, and thus filters low-confidence attention regions to generate refined attention maps that are then merged with original features to improve their representation ability. In addition, we also design an object-level loss to address the issues caused by touching glands. We evaluated our model against several recent gland segmentation methods and three typical semi-supervised methods on the GlaS and CRAG datasets. Our results not only demonstrate the effectiveness of the proposed due consistency module and Obj-Dice loss, but also indicate that the proposed I2CS model achieves state-of-the-art gland segmentation performance on both benchmarks.

Hip osteoarthritis: A novel network analysis of subchondral trabecular bone structures.

Hip osteoarthritis (HOA) is a degenerative joint disease that leads to the progressive destruction of subchondral bone and cartilage at the hip joint. Development of effective treatments for HOA remains an open problem, primarily due to the lack of knowledge of its pathogenesis and a typically late-stage diagnosis. We describe a novel network analysis methodology for microcomputed tomography (micro-CT) images of human trabecular bone. We explored differences between the trabecular bone microstructure of femoral heads with and without HOA. Large-scale automated extraction of the network formed by trabecular bone revealed significant network properties not previously reported for bone. Profound differences were discovered, particularly in the proximal third of the femoral head, where HOA networks demonstrated elevated numbers of edges, vertices, and graph components. When further differentiating healthy joint and HOA networks, the latter showed fewer small-world network properties, due to decreased clustering coefficient and increased characteristic path length. Furthermore, we found that HOA networks had reduced length of edges, indicating the formation of compressed trabecular structures. In order to assess our network approach, we developed a deep learning model for classifying HOA and control cases, and we fed it with two separate inputs: (i) micro-CT images of the trabecular bone, and (ii) the network extracted from them. The model with plain micro-CT images achieves 74.6% overall accuracy while the trained model with extracted networks attains 96.5% accuracy. We anticipate our findings to be a starting point for a novel description of bone microstructure in HOA, by considering the phenomenon from a graph theory viewpoint.