ABSTRACT
Background: Surgical action recognition is an essential technology in context-aware-based autonomous surgery, whereas the accuracy is limited by clinical dataset scale. Leveraging surgical videos from virtual reality (VR) simulations to research algorithms for the clinical domain application, also known as domain adaptation, can effectively reduce the cost of data acquisition and annotation, and protect patient privacy. Methods: We introduced a surgical domain adaptation method based on the contrastive language-image pretraining model (SDA-CLIP) to recognize cross-domain surgical action. Specifically, we utilized the Vision Transformer (ViT) and Transformer to extract video and text embeddings, respectively. Text embedding was developed as a bridge between VR and clinical domains. Inter- and intra-modality loss functions were employed to enhance the consistency of embeddings of the same class. Further, we evaluated our method on the MICCAI 2020 EndoVis Challenge SurgVisDom dataset. Results: Our SDA-CLIP achieved a weighted F1-score of 65.9% (+18.9%) on the hard domain adaptation task (trained only with VR data) and 84.4% (+4.4%) on the soft domain adaptation task (trained with VR and clinical-like data), which outperformed the first place team of the challenge by a significant margin. Conclusions: The proposed SDA-CLIP model can effectively extract video scene information and textual semantic information, which greatly improves the performance of cross-domain surgical action recognition. The code is available at https://github.com/Lycus99/SDA-CLIP.
ABSTRACT
BACKGROUND: To retrospectively analyze the pulmonary computed tomography (CT) characteristics and dynamic changes in the lungs of cured coronavirus disease 2019 (COVID-19) patients at discharge and reexamination. METHODS: A total of 155 cured COVID-19 patients admitted to designated hospitals in Yunnan Province, China, from February 1, 2020, to March 20, 2020, were included. All patients underwent pulmonary CT at discharge and at 2 weeks after discharge (during reexamination at hospital). A retrospective analysis was performed using these two pulmonary CT scans of the cured patients to observe changes in the number, distribution, morphology, and density of lesions. RESULTS: At discharge, the lung CT images of 15 cured patients showed no obvious lesions, while those of the remaining 140 patients showed different degrees of residual lesions. Patients with moderate disease mostly had multiple pulmonary lesions, mainly in the lower lobes of both lungs. At reexamination, the lung lesions in the patients with moderate disease had significantly improved (P<0.05), and the lung lesions in the patients with severe disease had partially improved, especially in patients with multi-lobe involvement (χ 2 =3.956, P<0.05). At reexamination, the lung lesions of patients with severe disease did not show significant changes (P>0.05). CONCLUSIONS: The pulmonary CT manifestations of cured COVID-19 patients had certain characteristics and variation patterns, providing a reference for the clinical evaluation of treatment efficacy and prognosis of patients.