LARLUS: laparoscopic augmented reality from laparoscopic ultrasound.

PURPOSE: This research endeavors to improve tumor localization in minimally invasive surgeries, a challenging task primarily attributable to the absence of tactile feedback and limited visibility. The conventional solution uses laparoscopic ultrasound (LUS) which has a long learning curve and is operator-dependent. METHODS: The proposed approach involves augmenting LUS images onto laparoscopic images to improve the surgeon's ability to estimate tumor and internal organ anatomy. This augmentation relies on LUS pose estimation and filtering. RESULTS: Experiments conducted with clinical data exhibit successful outcomes in both the registration and augmentation of LUS images onto laparoscopic images. Additionally, noteworthy results are observed in filtering, leading to reduced flickering in augmentations. CONCLUSION: The outcomes reveal promising results, suggesting the potential of LUS augmentation in surgical images to assist surgeons and serve as a training tool. We have used the LUS probe's shaft to disambiguate the rotational symmetry. However, in the long run, it would be desirable to find more convenient solutions.

Light modelling and calibration in laparoscopy.

PURPOSE: A better understanding of photometry in laparoscopic images can increase the reliability of computer-assisted surgery applications. Photometry requires modelling illumination, tissue reflectance and camera response. There exists a large variety of light models, but no systematic and reproducible evaluation. We present a review of light models in laparoscopic surgery, a unified calibration approach, an evaluation methodology, and a practical use of photometry. METHOD: We use images of a calibration checkerboard to calibrate the light models. We then use these models in a proposed dense stereo algorithm exploiting the shading and simultaneously extracting the tissue albedo, which we call dense shading stereo. The approach works with a broad range of light models, giving us a way to test their respective merits. RESULTS: We show that overly complex light models are usually not needed and that the light source position must be calibrated. We also show that dense shading stereo outperforms existing methods, in terms of both geometric and photometric errors, and achieves sub-millimeter accuracy. CONCLUSION: This work demonstrates the importance of careful light modelling and calibration for computer-assisted surgical applications. It gives guidelines on choosing the best performing light model.

An in vivo porcine dataset and evaluation methodology to measure soft-body laparoscopic liver registration accuracy with an extended algorithm that handles collisions.

PURPOSE: The registration of preoperative 3D images to intra-operative laparoscopic 2D images is one of the main concerns for augmented reality in computer-assisted surgery. For laparoscopic liver surgery, while several algorithms have been proposed, there is neither a public dataset nor a systematic evaluation methodology to quantitatively evaluate registration accuracy. METHOD: Our main contribution is to provide such a dataset with an in vivo porcine model. It is used to evaluate a state-of-the-art registration algorithm that is capable of simultaneous registration and soft-body collision reasoning. RESULTS: The dataset consists of 13 deformed liver states, with corresponding exploration videos and interventional CT acquisitions with 60 small artificial fiducials located on the surface of the liver and distributed within the parenchyma, where a precise registration is crucial for augmented reality. This dataset will be made public. Using this dataset, we show that collision reasoning improves performance of registration for strong deformation and independent lobe motion. CONCLUSION: This dataset addresses the lack of public datasets in this field. As an example of use, we present and evaluate a state-of-the-art energy-based approach and a novel extension that handles self-collisions.