Reducing annotating load: Active learning with synthetic images in surgical instrument segmentation.

Accurate instrument segmentation in the endoscopic vision of minimally invasive surgery is challenging due to complex instruments and environments. Deep learning techniques have shown competitive performance in recent years. However, deep learning usually requires a large amount of labeled data to achieve accurate prediction, which poses a significant workload. To alleviate this workload, we propose an active learning-based framework to generate synthetic images for efficient neural network training. In each active learning iteration, a small number of informative unlabeled images are first queried by active learning and manually labeled. Next, synthetic images are generated based on these selected images. The instruments and backgrounds are cropped out and randomly combined with blending and fusion near the boundary. The proposed method leverages the advantage of both active learning and synthetic images. The effectiveness of the proposed method is validated on two sinus surgery datasets and one intraabdominal surgery dataset. The results indicate a considerable performance improvement, especially when the size of the annotated dataset is small. All the code is open-sourced at: https://github.com/HaonanPeng/active_syn_generator.

Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation.

Accurate semantic image segmentation from medical imaging can enable intelligent vision-based assistance in robot-assisted minimally invasive surgery. The human body and surgical procedures are highly dynamic. While machine-vision presents a promising approach, sufficiently large training image sets for robust performance are either costly or unavailable. This work examines three novel generative adversarial network (GAN) methods of providing usable synthetic tool images using only surgical background images and a few real tool images. The best of these three novel approaches generates realistic tool textures while preserving local background content by incorporating both a style preservation and a content loss component into the proposed multi-level loss function. The approach is quantitatively evaluated, and results suggest that the synthetically generated training tool images enhance UNet tool segmentation performance. More specifically, with a random set of 100 cadaver and live endoscopic images from the University of Washington Sinus Dataset, the UNet trained with synthetically generated images using the presented method resulted in 35.7% and 30.6% improvement over using purely real images in mean Dice coefficient and Intersection over Union scores, respectively. This study is promising towards the use of more widely available and routine screening endoscopy to preoperatively generate synthetic training tool images for intraoperative UNet tool segmentation.

Characterizing Limits of Vision-Based Force Feedback in Simulated Surgical Tool-Tissue Interaction.

Haptic feedback can render real-time force interactions with computer simulated objects. In several telerobotic applications, it is desired that a haptic simulation reflects a physical task space or interaction accurately. This is particularly true when excessive applied force can result in disastrous consequences, as with the case of robot-assisted minimally invasive surgery (RMIS) and tissue damage. Since force cannot be directly measured in RMIS, non-contact methods are desired. A promising direction of non-contact force estimation involves the primary use of vision sensors to estimate deformation. However, the required fidelity of non-contact force rendering of deformable interaction to maintain surgical operator performance is not well established. This work attempts to empirically evaluate the degree to which haptic feedback may deviate from ground truth yet result in acceptable teleoperated performance in a simulated RMIS-based palpation task. A preliminary user-study is conducted to verify the utility of the simulation platform, and the results of this work have implications in haptic feedback for RMIS and inform guidelines for vision-based tool-tissue force estimation. An adaptive thresholding method is used to collect the minimum and maximum tolerable errors in force orientation and magnitude of presented haptic feedback to maintain sufficient performance.

Demethoxycurcumin modulates prostate cancer cell proliferation via AMPK-induced down-regulation of HSP70 and EGFR.

Curcumin (Cur), demethoxycurcumin (DMC), and bisdemethoxycurcumin (BDMC) are major forms of curcuminoids found in the rhizomes of turmeric. This study examined the effects of three curcuminoid analogues on prostate cancer cells. The results revealed that DMC demonstrated the most efficient cytotoxic effects on prostate cancer PC3 cells. DMC activated AMPK and in turn decreased the activity and/or expression of lipogenic enzymes, such as fatty acid synthase (FASN) and acetyl-CoA carboxylase (ACC). AICAR, an AMPK activator, and DMC down-regulated heat shock protein (HSP) 70 and increased the activity of the pro-apoptotic effector, caspase-3. In addition, DMC sustained epidermal growth factor receptor (EGFR) activation by suppressing the phosphatases PP2a and SHP-2. DMC also increased the interaction between EGFR and Cbl and induced the tyrosine phosphorylation of Cbl. The results suggest that DMC may have antitumor effects on prostate cancer cells via AMPK-induced down-regulation of HSP70 and EGFR.