CholecTriplet2022: Show me a tool and tell me the triplet - An endoscopic vision challenge for surgical action triplet detection.

Formalizing surgical activities as triplets of the used instruments, actions performed, and target anatomies is becoming a gold standard approach for surgical activity modeling. The benefit is that this formalization helps to obtain a more detailed understanding of tool-tissue interaction which can be used to develop better Artificial Intelligence assistance for image-guided surgery. Earlier efforts and the CholecTriplet challenge introduced in 2021 have put together techniques aimed at recognizing these triplets from surgical footage. Estimating also the spatial locations of the triplets would offer a more precise intraoperative context-aware decision support for computer-assisted intervention. This paper presents the CholecTriplet2022 challenge, which extends surgical action triplet modeling from recognition to detection. It includes weakly-supervised bounding box localization of every visible surgical instrument (or tool), as the key actors, and the modeling of each tool-activity in the form of triplet. The paper describes a baseline method and 10 new deep learning algorithms presented at the challenge to solve the task. It also provides thorough methodological comparisons of the methods, an in-depth analysis of the obtained results across multiple metrics, visual and procedural challenges; their significance, and useful insights for future research directions and applications in surgery.

CholecTriplet2021: A benchmark challenge for surgical action triplet recognition.

Context-aware decision support in the operating room can foster surgical safety and efficiency by leveraging real-time feedback from surgical workflow analysis. Most existing works recognize surgical activities at a coarse-grained level, such as phases, steps or events, leaving out fine-grained interaction details about the surgical activity; yet those are needed for more helpful AI assistance in the operating room. Recognizing surgical actions as triplets of combination delivers more comprehensive details about the activities taking place in surgical videos. This paper presents CholecTriplet2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos. The challenge granted private access to the large-scale CholecT50 dataset, which is annotated with action triplet information. In this paper, we present the challenge setup and the assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge. A total of 4 baseline methods from the challenge organizers and 19 new deep learning algorithms from the competing teams are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%. This study also analyzes the significance of the results obtained by the presented approaches, performs a thorough methodological comparison between them, in-depth result analysis, and proposes a novel ensemble method for enhanced recognition. Our analysis shows that surgical workflow analysis is not yet solved, and also highlights interesting directions for future research on fine-grained surgical activity recognition which is of utmost importance for the development of AI in surgery.

Learning to Generate Missing Pulse Sequence in MRI using Deep Convolution Neural Network Trained with Visual Turing Test.

Magnetic resonance imaging (MRI) is widely used in clinical applications due to its ability to acquire a wide variety of soft tissues using multiple pulse sequences. Each sequence provides information that generally complements the other. However, factors like an increase in scan time or contrast allergies impede imaging with numerous sequences. Synthesizing images of such non acquired sequences is a challenging proposition that can suffice for corrupted acquisition, fast reconstruction prior, super-resolution, etc. This manuscript employed a deep convolution neural network (CNN) to synthesize multiple missing pulse sequences of brain MRI with tumors. The CNN is an encoder-decoder-like network trained to minimize reconstruction mean square error (MSE) loss while maximizing the adversarial attack. It inflicts on a relativistic Visual Turing Test discriminator (rVTT). The approach is evaluated through experiments performed with the Brats2018 dataset, quantitative metrics viz. MSE, Structural Similarity Measure (SSIM), and Peak Signal to Noise Ratio (PSNR). The Radiologist and MR physicist performed the Turing test with 76% accuracy, demonstrating our approach's performance superiority over the prior art. We can synthesize MR images of missing pulse sequences at an inference cost of 350.71 GFlops/voxel through this approach.

CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentation.

Segmentation of abdominal organs has been a comprehensive, yet unresolved, research field for many years. In the last decade, intensive developments in deep learning (DL) introduced new state-of-the-art segmentation systems. Despite outperforming the overall accuracy of existing systems, the effects of DL model properties and parameters on the performance are hard to interpret. This makes comparative analysis a necessary tool towards interpretable studies and systems. Moreover, the performance of DL for emerging learning approaches such as cross-modality and multi-modal semantic segmentation tasks has been rarely discussed. In order to expand the knowledge on these topics, the CHAOS - Combined (CT-MR) Healthy Abdominal Organ Segmentation challenge was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI), 2019, in Venice, Italy. Abdominal organ segmentation from routine acquisitions plays an important role in several clinical applications, such as pre-surgical planning or morphological and volumetric follow-ups for various diseases. These applications require a certain level of performance on a diverse set of metrics such as maximum symmetric surface distance (MSSD) to determine surgical error-margin or overlap errors for tracking size and shape differences. Previous abdomen related challenges are mainly focused on tumor/lesion detection and/or classification with a single modality. Conversely, CHAOS provides both abdominal CT and MR data from healthy subjects for single and multiple abdominal organ segmentation. Five different but complementary tasks were designed to analyze the capabilities of participating approaches from multiple perspectives. The results were investigated thoroughly, compared with manual annotations and interactive methods. The analysis shows that the performance of DL models for single modality (CT / MR) can show reliable volumetric analysis performance (DICE: 0.98 ± 0.00 / 0.95 ± 0.01), but the best MSSD performance remains limited (21.89 ± 13.94 / 20.85 ± 10.63 mm). The performances of participating models decrease dramatically for cross-modality tasks both for the liver (DICE: 0.88 ± 0.15 MSSD: 36.33 ± 21.97 mm). Despite contrary examples on different applications, multi-tasking DL models designed to segment all organs are observed to perform worse compared to organ-specific ones (performance drop around 5%). Nevertheless, some of the successful models show better performance with their multi-organ versions. We conclude that the exploration of those pros and cons in both single vs multi-organ and cross-modality segmentations is poised to have an impact on further research for developing effective algorithms that would support real-world clinical applications. Finally, having more than 1500 participants and receiving more than 550 submissions, another important contribution of this study is the analysis on shortcomings of challenge organizations such as the effects of multiple submissions and peeking phenomenon.

A Two-Stage Multiple Instance Learning Framework for the Detection of Breast Cancer in Mammograms.

Mammograms are commonly employed in the large scale screening of breast cancer which is primarily characterized by the presence of malignant masses. However, automated image-level detection of malignancy is a challenging task given the small size of the mass regions and difficulty in discriminating between malignant, benign mass and healthy dense fibro-glandular tissue. To address these issues, we explore a two-stage Multiple Instance Learning (MIL) framework. A Convolutional Neural Network (CNN) is trained in the first stage to extract local candidate patches in the mammograms that may contain either a benign or malignant mass. The second stage employs a MIL strategy for an image level benign vs. malignant classification. A global image-level feature is computed as a weighted average of patch-level features learned using a CNN. Our method performed well on the task of localization of masses with an average Precision/Recall of 0.76/0.80 and achieved an average AUC of 0.91 on the image-level classification task using a five-fold cross-validation on the INbreast dataset. Restricting the MIL only to the candidate patches extracted in Stage 1 led to a significant improvement in classification performance in comparison to a dense extraction of patches from the entire mammogram.

Lung Segmentation and Nodule Detection in Computed Tomography Scan using a Convolutional Neural Network Trained Adversarially using Turing Test Loss.

Lung cancer is the most common form of cancer found worldwide with a high mortality rate. Early detection of pulmonary nodules by screening with a low-dose computed tomography (CT) scan is crucial for its effective clinical management. Nodules which are symptomatic of malignancy occupy about 0.0125 - 0.025% of volume in a CT scan of a patient. Manual screening of all slices is a tedious task and presents a high risk of human errors. To tackle this problem we propose a computationally efficient two stage framework. In the first stage, a convolutional neural network (CNN) trained adversarially using Turing test loss segments the lung region. In the second stage, patches sampled from the segmented region are then classified to detect the presence of nodules. The proposed method is experimentally validated on the LUNA16 challenge dataset with a dice coefficient of 0.984±0.0007 for 10-fold cross-validation.

Evaluation of Commonly Used Algorithms for Thyroid Ultrasound Images Segmentation and Improvement Using Machine Learning Approaches.

The thyroid is one of the largest endocrine glands in the human body, which is involved in several body mechanisms like controlling protein synthesis and the body's sensitivity to other hormones and use of energy sources. Hence, it is of prime importance to track the shape and size of thyroid over time in order to evaluate its state. Thyroid segmentation and volume computation are important tools that can be used for thyroid state tracking assessment. Most of the proposed approaches are not automatic and require long time to correctly segment the thyroid. In this work, we compare three different nonautomatic segmentation algorithms (i.e., active contours without edges, graph cut, and pixel-based classifier) in freehand three-dimensional ultrasound imaging in terms of accuracy, robustness, ease of use, level of human interaction required, and computation time. We figured out that these methods lack automation and machine intelligence and are not highly accurate. Hence, we implemented two machine learning approaches (i.e., random forest and convolutional neural network) to improve the accuracy of segmentation as well as provide automation. This comparative study intends to discuss and analyse the advantages and disadvantages of different algorithms. In the last step, the volume of the thyroid is computed using the segmentation results, and the performance analysis of all the algorithms is carried out by comparing the segmentation results with the ground truth.

Learning of speckle statistics for in vivo and noninvasive characterization of cutaneous wound regions using laser speckle contrast imaging.

Laser speckle contrast imaging (LSCI) provides a noninvasive and cost effective solution for in vivo monitoring of blood flow. So far, most of the researches consider changes in speckle pattern (i.e. correlation time of speckle intensity fluctuation), account for relative change in blood flow during abnormal conditions. This paper introduces an application of LSCI for monitoring wound progression and characterization of cutaneous wound regions on mice model. Speckle images are captured on a tumor wound region at mice leg in periodic interval. Initially, raw speckle images are converted to their corresponding contrast images. Functional characterization begins with first segmenting the affected area using k-means clustering, taking wavelet energies in a local region as feature set. In the next stage, different regions in wound bed are clustered based on progressive and non-progressive nature of tissue properties. Changes in contrast due to heterogeneity in tissue structure and functionality are modeled using LSCI speckle statistics. Final characterization is achieved through supervised learning of these speckle statistics using support vector machine. On cross evaluation with mice model experiment, the proposed approach classifies the progressive and non-progressive wound regions with an average sensitivity of 96.18%, 97.62% and average specificity of 97.24%, 96.42% respectively. The clinical information yield with this approach is validated with the conventional immunohistochemistry result of wound to justify the ability of LSCI for in vivo, noninvasive and periodic assessment of wounds.

Modulating prime molecular expressions and in vitro wound healing rate in keratinocyte (HaCaT) population under characteristic honey dilutions.

ETHNOPHARMACOLOGY RELEVANCE: In traditional medicines honey is known for healing efficacy and vividly used as "Anupan" in Ayurvedic medicines appreciating roles in dilutions. Validating efficacy of physico-chemically characterized honey in dilutions, studies on in vitro wound healing and attainment of cellular confluence epithelial cells including expressions of cardinal genes is crucial. To evaluate effects of characterized honey in varied dilutions on cellular viability, in vitro wound healing and modulation of prime epithelial gene expressions. MATERIALS AND METHODS: Six Indian honey-samples from different sources were physico-chemically characterized and optimal one was explored in dilutions (v/v%) through in vitro studies on human epithelial (HaCaT) cells for viability, wound healing and expressions of genes p63, E-cadherin, ß-catenin, GnT-III and GnT-V. RESULTS: Studied honey samples (i.e. A-F) depicted range of pH (2-4), water (12.48-23.95), electrical conductivity (2.57-14.34), carbohydrate (68.73-98.65), protein (.316-5.36) and antioxidant potential. Though sample A and F showed physico-chemical proximity, but overall bio-impact of the earlier was better, thus studied in 8-.1% (v/v) dilution range. Four dilutions (.01, .04, .1, .25 v/v%) augmented cellular viability but in vitro wound healing was fastest (p<.05) under .1%. Such efficacy was further documented for p63 up-regulation by immunocytochemistry and mRNA studies. The E-cadherin and ß-catenin mRNA-expressions were also up-regulated and their proteins were predominantly cytoplasmic. E-cadherin up-regulation was corroborative with down-regulation and up-regulation of GnT-III and GnT-V respectively. CONCLUSION: Present study illustrated efficacy of particular honey dilution (.1%) with characteristic free radical scavenging activity in facilitating cell proliferation and attainment of confluence towards faster wound healing and modulation of cardinal epithelial genes (viz. p63, E-cadherin, ß-catenin, Gnt-III and V).

Breast fine needle aspiration cytology practices and commonly perceived diagnostic significance of cytological features: A pan- India survey.

BACKGROUND: Methodical and meticulous understanding of clinico-pathological procedures and decision making process of cancer diagnosis and identification of aspects that are well-suited for computer-aided analysis are first steps toward development of assistive computational tool for analysis of breast fine needle aspiration cytology (FNAC) slides. AIMS: To identify variables in practice of FNAC as used for diagnosis of breast lesions and commonly perceived diagnostic significance of cytological features for diagnosis of benign or malignant condition of breast lesions. MATERIALS AND METHODS: An India-wide questionnaire-based survey of cytopathologists/pathologists' breast FNAC reporting practices and their opinion on diagnostic significance of cytological features in diagnosis of benign or malignant nature of breast lesion were conducted. RESULTS: Fifty-one experts working with various medical education institutes (~52% of participants), oncological tertiary care centers (~28%) and primary care centers/private diagnostic pathology laboratories (~20%) spread over 13 states of India have participated in the survey. Constants and variables observed in clinico-cytopathological practices and combined opinion of the participants on diagnostic significance of cytological features are presented here. CONCLUSIONS: There exist analogous as well as varied components in clinico-pathological procedures and diagnostic interpretation by individuals. These constants and variables in the practice of breast FNAC should be considered, when drawing up specifications for an assistive computational tool for analysis of breast FNAC slides. The estimate for commonly perceived significance of cytological features obtained through this study will help in their selection for computer-aided analysis of breast FNAC slides and further in selection of corresponding feature quantification techniques.