Structured light for touchless 3D registration in video-based surgical navigation.

PURPOSE: Arthroscopic surgery, with its inherent difficulties on visibility and maneuverability inside the joint, poses significant challenges to surgeons. Video-based surgical navigation (VBSN) has proven to have clinical benefits in arthroscopy but relies on a time-consuming and challenging surface digitization using a touch probe to accomplish registration of intraoperative data with preoperative anatomical models. This paper presents an off-the-shelf laser scanner for noninvasive registration that enables an increased area of reachable region. METHODS: Our solution uses a standard arthroscope and a light projector with visual markers for real-time extrinsic calibration. Nevertheless, the shift from a touch probe to a laser scanner introduces a new challenge-the presence of a significant amount of outliers resulting from the reconstruction of nonrigid structures. To address this issue, we propose to identify the structures of interest prior to reconstruction using a deep learning-based semantic segmentation technique. RESULTS: Experimental validation using knee and hip phantoms, as well as ex-vivo data, assesses the laser scanner's effectiveness. The integration of the segmentation model improves results in ex-vivo experiments by mitigating outliers. Specifically, the laser scanner with the segmentation model achieves registration errors below 2.2 mm, with the intercondylar region exhibiting errors below 1 mm. In experiments with phantoms, the errors are always below 1 mm. CONCLUSION: The results show the viability of integrating the laser scanner with VBSN as a noninvasive and potential alternative to traditional methods by overcoming surface digitization challenges and expanding the reachable region. Future efforts aim to improve hardware to further optimize performance and applicability in complex procedures.

Video-based computer navigation in knee arthroscopy for patient-specific ACL reconstruction.

PURPOSE: The anterior cruciate ligament tear is a common medical condition that is treated using arthroscopy by pulling a tissue graft through a tunnel opened with a drill. The correct anatomical position and orientation of this tunnel are crucial for knee stability, and drilling an adequate bone tunnel is the most technically challenging part of the procedure. This paper presents the first guidance system based solely on intra-operative video for guiding the drilling of the tunnel. METHODS: Our solution uses small, easily recognizable visual markers that are attached to the bone and tools for estimating their relative pose. A recent registration algorithm is employed for aligning a pre-operative image of the patient's anatomy with a set of contours reconstructed by touching the bone surface with an instrumented tool. RESULTS: Experimental validation using ex-vivo data shows that the method enables the accurate registration of the pre-operative model with the bone, providing useful information for guiding the surgeon during the medical procedure. Experiments also demonstrate that the guided drilling of the tunnel leads to errors as low as 2.5 mm in the footprint and [Formula: see text] in orientation, which compares favourably to other works in the field. CONCLUSION: The high accuracy and short time overhead evinced by the experimental validation combined with no additional incisions or capital equipment make this video-based computer-aided arthroscopy solution an appealing alternative to the existing approaches.

Deep segmentation leverages geometric pose estimation in computer-aided total knee arthroplasty.

Knee arthritis is a common joint disease that usually requires a total knee arthroplasty. There are multiple surgical variables that have a direct impact on the correct positioning of the implants, and an optimal combination of all these variables is the most challenging aspect of the procedure. Usually, preoperative planning using a computed tomography scan or magnetic resonance imaging helps the surgeon in deciding the most suitable resections to be made. This work is a proof of concept for a navigation system that supports the surgeon in following a preoperative plan. Existing solutions require costly sensors and special markers, fixed to the bones using additional incisions, which can interfere with the normal surgical flow. In contrast, the authors propose a computer-aided system that uses consumer RGB and depth cameras and do not require additional markers or tools to be tracked. They combine a deep learning approach for segmenting the bone surface with a recent registration algorithm for computing the pose of the navigation sensor with respect to the preoperative 3D model. Experimental validation using ex-vivo data shows that the method enables contactless pose estimation of the navigation sensor with the preoperative model, providing valuable information for guiding the surgeon during the medical procedure.

Automatic Camera Calibration Using Multiple Sets of Pairwise Correspondences.

We propose a new method to add an uncalibrated node into a network of calibrated cameras using only pairwise point correspondences. While previous methods perform this task using triple correspondences, these are often difficult to establish when there is limited overlap between different views. In such challenging cases we must rely on pairwise correspondences and our solution becomes more advantageous. Our method includes an 11-point minimal solution for the intrinsic and extrinsic calibration of a camera from pairwise correspondences with other two calibrated cameras, and a new inlier selection framework that extends the traditional RANSAC family of algorithms to sampling across multiple datasets. Our method is validated on different application scenarios where a lack of triple correspondences might occur: addition of a new node to a camera network; calibration and motion estimation of a moving camera inside a camera network; and addition of views with limited overlap to a Structure-from-Motion model.

Continuous zoom calibration by tracking salient points in endoscopic video.

Many image-based systems for aiding the surgeon during minimally invasive surgery require the endoscopic camera to be calibrated at all times. This article proposes a method for accomplishing this goal whenever the camera has optical zoom and the focal length changes during the procedure. Our solution for online calibration builds on recent developments in tracking salient points using differential image alignment, is well suited for continuous operation, and makes no assumptions about the camera motion or scene rigidity. Experimental validation using both a phantom model and in vivo data shows that the method enables accurate estimation of focal length when the zoom varies, avoiding the need to explicitly recalibrate during surgery. To the best of our knowledge this the first work proposing a practical solution for online zoom calibration in the operation room.

A minimal solution for the extrinsic calibration of a camera and a laser-rangefinder.

This paper presents a new algorithm for the extrinsic calibration of a perspective camera and an invisible 2D laser-rangefinder (LRF). The calibration is achieved by freely moving a checkerboard pattern in order to obtain plane poses in camera coordinates and depth readings in the LRF reference frame. The problem of estimating the rigid displacement between the two sensors is formulated as one of registering a set of planes and lines in the 3D space. It is proven for the first time that the alignment of three plane-line correspondences has at most eight solutions that can be determined by solving a standard p3p problem and a linear system of equations. This leads to a minimal closed-form solution for the extrinsic calibration that can be used as hypothesis generator in a RANSAC paradigm. Our calibration approach is validated through simulation and real experiments that show the superiority with respect to the current state-of-the-art method requiring a minimum of five input planes.

A new solution for camera calibration and real-time image distortion correction in medical endoscopy-initial technical evaluation.

Medical endoscopy is used in a wide variety of diagnostic and surgical procedures. These procedures are renowned for the difficulty of orienting the camera and instruments inside the human body cavities. The small size of the lens causes radial distortion of the image, which hinders the navigation process and leads to errors in depth perception and object morphology. This article presents a complete software-based system to calibrate and correct the radial distortion in clinical endoscopy in real time. Our system can be used with any type of medical endoscopic technology, including oblique-viewing endoscopes and HD image acquisition. The initial camera calibration is performed in an unsupervised manner from a single checkerboard pattern image. For oblique-viewing endoscopes the changes in calibration during operation are handled by a new adaptive camera projection model and an algorithm that infer the rotation of the probe lens using only image information. The workload is distributed across the CPU and GPU through an optimized CPU+GPU hybrid solution. This enables real-time performance, even for HD video inputs. The system is evaluated for different technical aspects, including accuracy of modeling and calibration, overall robustness, and runtime profile. The contributions are highly relevant for applications in computer-aided surgery and image-guided intervention such as improved visualization by image warping, 3-D modeling, and visual SLAM.

Geometric properties of central catadioptric line images and their application in calibration.

In central catadioptric systems, lines in a scene are projected to conic curves in the image. This work studies the geometry of the central catadioptric projection of lines and its use in calibration. It is shown that the conic curves where the lines are mapped possess several projective invariant properties. From these properties, it follows that any central catadioptric system can be fully calibrated from an image of three or more lines. The image of the absolute conic, the relative pose between the camera and the mirror, and the shape of the reflective surface can be recovered using a geometric construction based on the conic loci where the lines are projected. This result is valid for any central catadioptric system and generalizes previous results for paracatadioptric sensors. Moreover, it is proven that systems with a hyperbolic/elliptical mirror can be calibrated from the image of two lines. If both the shape and the pose of the mirror are known, then two line images are enough to determine the image of the absolute conic encoding the camera's intrinsic parameters. The sensitivity to errors is evaluated and the approach is used to calibrate a real camera.