Optimizing latent graph representations of surgical scenes for unseen domain generalization.

PURPOSE: Advances in deep learning have resulted in effective models for surgical video analysis; however, these models often fail to generalize across medical centers due to domain shift caused by variations in surgical workflow, camera setups, and patient demographics. Recently, object-centric learning has emerged as a promising approach for improved surgical scene understanding, capturing and disentangling visual and semantic properties of surgical tools and anatomy to improve downstream task performance. In this work, we conduct a multicentric performance benchmark of object-centric approaches, focusing on critical view of safety assessment in laparoscopic cholecystectomy, then propose an improved approach for unseen domain generalization. METHODS: We evaluate four object-centric approaches for domain generalization, establishing baseline performance. Next, leveraging the disentangled nature of object-centric representations, we dissect one of these methods through a series of ablations (e.g., ignoring either visual or semantic features for downstream classification). Finally, based on the results of these ablations, we develop an optimized method specifically tailored for domain generalization, LG-DG, that includes a novel disentanglement loss function. RESULTS: Our optimized approach, LG-DG, achieves an improvement of 9.28% over the best baseline approach. More broadly, we show that object-centric approaches are highly effective for domain generalization thanks to their modular approach to representation learning. CONCLUSION: We investigate the use of object-centric methods for unseen domain generalization, identify method-agnostic factors critical for performance, and present an optimized approach that substantially outperforms existing methods.

Latent Graph Representations for Critical View of Safety Assessment.

Assessing the critical view of safety in laparoscopic cholecystectomy requires accurate identification and localization of key anatomical structures, reasoning about their geometric relationships to one another, and determining the quality of their exposure. Prior works have approached this task by including semantic segmentation as an intermediate step, using predicted segmentation masks to then predict the CVS. While these methods are effective, they rely on extremely expensive ground-truth segmentation annotations and tend to fail when the predicted segmentation is incorrect, limiting generalization. In this work, we propose a method for CVS prediction wherein we first represent a surgical image using a disentangled latent scene graph, then process this representation using a graph neural network. Our graph representations explicitly encode semantic information - object location, class information, geometric relations - to improve anatomy-driven reasoning, as well as visual features to retain differentiability and thereby provide robustness to semantic errors. Finally, to address annotation cost, we propose to train our method using only bounding box annotations, incorporating an auxiliary image reconstruction objective to learn fine-grained object boundaries. We show that our method not only outperforms several baseline methods when trained with bounding box annotations, but also scales effectively when trained with segmentation masks, maintaining state-of-the-art performance.

Dissecting self-supervised learning methods for surgical computer vision.

The field of surgical computer vision has undergone considerable breakthroughs in recent years with the rising popularity of deep neural network-based methods. However, standard fully-supervised approaches for training such models require vast amounts of annotated data, imposing a prohibitively high cost; especially in the clinical domain. Self-Supervised Learning (SSL) methods, which have begun to gain traction in the general computer vision community, represent a potential solution to these annotation costs, allowing to learn useful representations from only unlabeled data. Still, the effectiveness of SSL methods in more complex and impactful domains, such as medicine and surgery, remains limited and unexplored. In this work, we address this critical need by investigating four state-of-the-art SSL methods (MoCo v2, SimCLR, DINO, SwAV) in the context of surgical computer vision. We present an extensive analysis of the performance of these methods on the Cholec80 dataset for two fundamental and popular tasks in surgical context understanding, phase recognition and tool presence detection. We examine their parameterization, then their behavior with respect to training data quantities in semi-supervised settings. Correct transfer of these methods to surgery, as described and conducted in this work, leads to substantial performance gains over generic uses of SSL - up to 7.4% on phase recognition and 20% on tool presence detection - as well as state-of-the-art semi-supervised phase recognition approaches by up to 14%. Further results obtained on a highly diverse selection of surgical datasets exhibit strong generalization properties. The code is available at https://github.com/CAMMA-public/SelfSupSurg.

CholecTriplet2022: Show me a tool and tell me the triplet - An endoscopic vision challenge for surgical action triplet detection.

Formalizing surgical activities as triplets of the used instruments, actions performed, and target anatomies is becoming a gold standard approach for surgical activity modeling. The benefit is that this formalization helps to obtain a more detailed understanding of tool-tissue interaction which can be used to develop better Artificial Intelligence assistance for image-guided surgery. Earlier efforts and the CholecTriplet challenge introduced in 2021 have put together techniques aimed at recognizing these triplets from surgical footage. Estimating also the spatial locations of the triplets would offer a more precise intraoperative context-aware decision support for computer-assisted intervention. This paper presents the CholecTriplet2022 challenge, which extends surgical action triplet modeling from recognition to detection. It includes weakly-supervised bounding box localization of every visible surgical instrument (or tool), as the key actors, and the modeling of each tool-activity in the form of triplet. The paper describes a baseline method and 10 new deep learning algorithms presented at the challenge to solve the task. It also provides thorough methodological comparisons of the methods, an in-depth analysis of the obtained results across multiple metrics, visual and procedural challenges; their significance, and useful insights for future research directions and applications in surgery.

Digital signatures for early traumatic brain injury outcome prediction in the intensive care unit.

Traumatic brain injury (TBI) is a leading neurological cause of death and disability across the world. Early characterization of TBI severity could provide a window for therapeutic intervention and contribute to improved outcome. We hypothesized that granular electronic health record data available in the first 24 h following admission to the intensive care unit (ICU) can be used to differentiate outcomes at discharge. Working from two ICU datasets we focused on patients with a primary admission diagnosis of TBI whose length of stay in ICU was ≥ 24 h (N = 1689 and 127). Features derived from clinical, laboratory, medication, and physiological time series data in the first 24 h after ICU admission were used to train elastic-net regularized Generalized Linear Models for the prediction of mortality and neurological function at ICU discharge. Model discrimination, determined by area under the receiver operating characteristic curve (AUC) analysis, was 0.903 and 0.874 for mortality and neurological function, respectively. Model performance was successfully validated in an external dataset (AUC 0.958 and 0.878 for mortality and neurological function, respectively). These results demonstrate that computational analysis of data routinely collected in the first 24 h after admission accurately and reliably predict discharge outcomes in ICU stratum TBI patients.

Outcomes in Lung Cancer: 9-Year Experience From a Tertiary Cancer Center in India.

PURPOSE: Lung cancer is the most common cause of cancer mortality in the world. There are limited studies on survival outcomes of lung cancer in developing countries such as India. This study analyzed the outcomes of patients with lung cancer who underwent treatment at Cancer Institute (WIA), Chennai, India, between 2006 and 2015 to determine survival outcomes and identify prognostic factors. PATIENTS AND METHODS: In all, 678 patients with lung cancer underwent treatment. Median age was 58 years, and 91% of patients had non-small-cell lung cancer (NSCLC). Testing for epidermal growth factor receptor mutation was performed in 132 of 347 patients and 61 (46%) were positive. RESULTS: Median progression-free survival was 6.9 months and overall survival (OS) was 7.6 months for patients with NSCLC. Median progression-free survival was 6 months and OS was 7.2 months for patients with small-cell lung cancer. On multivariable analysis, the factors found to be significantly associated with inferior OS in NSCLC included nonadenocarcinoma histology, performance status more than 2, and stage. In small-cell lung cancer, younger age and earlier stage at presentation showed significantly better survival. CONCLUSION: Our study highlights the challenges faced in treating lung cancer in India. Although median survival in advanced-stage lung cancer is still poor, strategies such as personalized medicine and use of second-line and maintenance chemotherapy may significantly improve the survival in patients with advanced-stage lung cancer in developing countries.