Robotic middle ear access for cochlear implantation: First in man.

To demonstrate the feasibility of robotic middle ear access in a clinical setting, nine adult patients with severe-to-profound hearing loss indicated for cochlear implantation were included in this clinical trial. A keyhole access tunnel to the tympanic cavity and targeting the round window was planned based on preoperatively acquired computed tomography image data and robotically drilled to the level of the facial recess. Intraoperative imaging was performed to confirm sufficient distance of the drilling trajectory to relevant anatomy. Robotic drilling continued toward the round window. The cochlear access was manually created by the surgeon. Electrode arrays were inserted through the keyhole tunnel under microscopic supervision via a tympanomeatal flap. All patients were successfully implanted with a cochlear implant. In 9 of 9 patients the robotic drilling was planned and performed to the level of the facial recess. In 3 patients, the procedure was reverted to a conventional approach for safety reasons. No change in facial nerve function compared to baseline measurements was observed. Robotic keyhole access for cochlear implantation is feasible. Further improvements to workflow complexity, duration of surgery, and usability including safety assessments are required to enable wider adoption of the procedure.

Electrical Impedance to Assess Facial Nerve Proximity During Robotic Cochlear Implantation.

Reported studies pertaining to needle guidance suggest that tissue impedance available from neuromonitoring systems can be used to discriminate nerve tissue proximity. In this pilot study, the existence of a relationship between intraoperative electrical impedance and tissue density, estimated from computer tomography (CT) images, is evaluated in the mastoid bone of in vivo sheep. In five subjects, nine trajectories were drilled using an image-guided surgical robot. Per trajectory, five measurement points near the facial nerve were accessed and electrical impedance was measured (≤1 KHz) using a multipolar electrode probe. Micro-CT was used postoperatively to measure the distances from the drilled trajectories to the facial nerve. Tissue density was determined from coregistered preoperative CT images and, following sensitivity field modeling of the measuring tip, tissue resistivity was calculated. The relationship between impedance and density was determined for 29 trajectories passing or intersecting the facial nerve. A monotonic decrease in impedance magnitude was observed in all trajectories with a drill axis intersecting the facial nerve. Mean tissue densities intersecting with the facial nerve (971-1161 HU) were different (p <0.01) from those along safe trajectories passing the nerve (1194-1449 HU). However, mean resistivity values of trajectories intersecting the facial nerve (14-24 Ωm) were similar to those of safe passing trajectories (17-23 Ωm). The determined relationship between tissue density and electrical impedance during neuromonitoring of the facial nerve suggests that impedance spectroscopy may be used to increase the accuracy of tissue discrimination, and ultimately improve nerve safety distance assessment in the future.

Neuromonitoring During Robotic Cochlear Implantation: Initial Clinical Experience.

During robotic cochlear implantation a drill trajectory often passes at submillimeter distances from the facial nerve due to close lying critical anatomy of the temporal bone. Additional intraoperative safety mechanisms are thus required to ensure preservation of this vital structure in case of unexpected navigation system error. Electromyography based nerve monitoring is widely used to aid surgeons in localizing vital nerve structures at risk of injury during surgery. However, state of the art neuromonitoring systems, are unable to discriminate facial nerve proximity within submillimeter ranges. Previous work demonstrated the feasibility of utilizing combinations of monopolar and bipolar stimulation threshold measurements to discretize facial nerve proximity with greater sensitivity and specificity, enabling discrimination between safe (> 0.4 mm) and unsafe (< 0.1 mm) trajectories during robotic cochlear implantation (in vivo animal model). Herein, initial clinical validation of the determined stimulation protocol and nerve proximity analysis integrated into an image guided system for safety measurement is presented. Stimulation thresholds and corresponding nerve proximity values previously determined from an animal model have been validated in a first-in-man clinical trial of robotic cochlear implantation. Measurements performed automatically at preoperatively defined distances from the facial nerve were used to determine safety of the drill trajectory intraoperatively. The presented system and automated analysis correctly determined sufficient safety distance margins (> 0.4 mm) to the facial nerve in all cases.

Highly Accurate Facial Nerve Segmentation Refinement From CBCT/CT Imaging Using a Super-Resolution Classification Approach.

Facial nerve segmentation is of considerable importance for preoperative planning of cochlear implantation. However, it is strongly influenced by the relatively low resolution of the cone-beam computed tomography (CBCT) images used in clinical practice. In this paper, we propose a super-resolution classification method, which refines a given initial segmentation of the facial nerve to a subvoxel classification level from CBCT/CT images. The super-resolution classification method learns the mapping from low-resolution CBCT/CT images to high-resolution facial nerve label images, obtained from manual segmentation on micro-CT images. We present preliminary results on dataset, 15 ex vivo samples scanned including pairs of CBCT/CT scans and high-resolution micro-CT scans, with a leave-one-out evaluation, and manual segmentations on micro-CT images as ground truth. Our experiments achieved a segmentation accuracy with a Dice coefficient of , surface-to-surface distance of , and Hausdorff distance of . We compared the proposed technique to two other semi-automated segmentation software tools, ITK-SNAP and GeoS, and show the ability of the proposed approach to yield subvoxel levels of accuracy in delineating the facial nerve.

Real-Time Prediction of Temperature Elevation During Robotic Bone Drilling Using the Torque Signal.

Bone drilling is a surgical procedure commonly required in many surgical fields, particularly orthopedics, dentistry and head and neck surgeries. While the long-term effects of thermal bone necrosis are unknown, the thermal damage to nerves in spinal or otolaryngological surgeries might lead to partial paralysis. Previous models to predict the temperature elevation have been suggested, but were not validated or have the disadvantages of computation time and complexity which does not allow real time predictions. Within this study, an analytical temperature prediction model is proposed which uses the torque signal of the drilling process to model the heat production of the drill bit. A simple Green's disk source function is used to solve the three dimensional heat equation along the drilling axis. Additionally, an extensive experimental study was carried out to validate the model. A custom CNC-setup with a load cell and a thermal camera was used to measure the axial drilling torque and force as well as temperature elevations. Bones with different sets of bone volume fraction were drilled with two drill bits ([Formula: see text]1.8 mm and [Formula: see text]2.5 mm) and repeated eight times. The model was calibrated with 5 of 40 measurements and successfully validated with the rest of the data ([Formula: see text]C). It was also found that the temperature elevation can be predicted using only the torque signal of the drilling process. In the future, the model could be used to monitor and control the drilling process of surgeries close to vulnerable structures.

Population Statistics Approach for Safety Assessment in Robotic Cochlear Implantation.

HYPOTHESIS: Descriptive statistics with respect to patient anatomy and image guidance accuracy can be used to assess the effectiveness of any system for minimally invasive cochlear implantation, on both an individual patient and wider population level. BACKGROUND: Minimally invasive cochlear implantation involves the drilling of a tunnel from the surface of the mastoid to cochlea, with the trajectory passing through the facial recess. The facial recess anatomy constrains the drilling path and places prohibitive accuracy requirements on the used system. Existing single thresholds are insufficient for assessing the effectiveness of these systems. METHODS: A statistical model of the anatomical situation encountered during minimally invasive drilling of the mastoid for cochlear implantation was developed. A literature review was performed to determine the statistical distribution of facial recess width; these values were confirmed through facial recess measurements on computed tomography (CT) data. Based on the accuracy of a robotic system developed by the authors, the effect of variation of system accuracy, precision, and tunnel diameter examined with respect to the potential treatable portion of the population. RESULTS: A facial recess diameter of 2.54 ±â€Š0.51 mm (n = 74) was determined from a review of existing literature; subsequent measurements on CT data revealed a facial recess diameter of 2.54 ±â€Š0.5 mm (n = 23). The developed model demonstrated the effects of varying accuracy on the treatable portion of the population. CONCLUSIONS: The presented model allows the assessment of the applicability of a system on a wider population scale beyond examining only the system's ability to reach an arbitrary threshold accuracy.

Robotic cochlear implantation: surgical procedure and first clinical experience.

CONCLUSION: A system for robotic cochlear implantation (rCI) has been developed and a corresponding surgical workflow has been described. The clinical feasibility was demonstrated through the conduction of a safe and effective rCI procedure. OBJECTIVES: To define a clinical workflow for rCI and demonstrate its feasibility, safety, and effectiveness within a clinical setting. METHOD: A clinical workflow for use of a previously described image guided surgical robot system for rCI was developed. Based on pre-operative images, a safe drilling tunnel targeting the round window was planned and drilled by the robotic system. Intra-operatively the drill path was assessed using imaging and sensor-based data to confirm the proximity of the facial nerve. Electrode array insertion was manually achieved under microscope visualization. Electrode array placement, structure preservation, and the accuracy of the drilling and of the safety mechanisms were assessed on post-operative CT images. RESULTS: Robotic drilling was conducted with an accuracy of 0.2 mm and safety mechanisms predicted proximity of the nerves to within 0.1 mm. The approach resulted in a minimal mastoidectomy and minimal incisions. Manual electrode array insertion was successfully performed through the robotically drilled tunnel. The procedure was performed without complications, and all surrounding structures were preserved.

In-Vivo Electrical Impedance Measurement in Mastoid Bone.

Nerve monitoring is a safety mechanism to detect the proximity between surgical instruments and important nerves during surgical bone preparation. In temporal bone, this technique is highly specific and sensitive at distances below 0.1 mm, but remains unreliable for distances above this threshold. A deeper understanding of the patient-specific bone electric properties is required to improve this range of detection. A sheep animal model has been used to characterize bone properties in vivo. Impedance measurements have been performed at low frequencies (<1 kHz) between two electrodes placed inside holes drilled into the sheep mastoid bone. An electric circuit composed of a resistor and a Fricke constant phase element was able to accurately describe the experimental measurements. Bone resistivity was shown to be linearly dependent on the inter-electrode distance and the local bone density. Based on this model, the amount of bone material between the electrodes could be predicted with an error of 0.7 mm. Our results indicate that bone could be described as an ideal resistor while the electrochemical processes at the electrode-tissue interface are characterized by a constant phase element. These results should help increasing the safety of surgical drilling procedures by better predicting the distance to critical nerve structures.

Temperature Prediction Model for Bone Drilling Based on Density Distribution and In Vivo Experiments for Minimally Invasive Robotic Cochlear Implantation.

Surgical robots have been proposed ex vivo to drill precise holes in the temporal bone for minimally invasive cochlear implantation. The main risk of the procedure is damage of the facial nerve due to mechanical interaction or due to temperature elevation during the drilling process. To evaluate the thermal risk of the drilling process, a simplified model is proposed which aims to enable an assessment of risk posed to the facial nerve for a given set of constant process parameters for different mastoid bone densities. The model uses the bone density distribution along the drilling trajectory in the mastoid bone to calculate a time dependent heat production function at the tip of the drill bit. Using a time dependent moving point source Green's function, the heat equation can be solved at a certain point in space so that the resulting temperatures can be calculated over time. The model was calibrated and initially verified with in vivo temperature data. The data was collected in minimally invasive robotic drilling of 12 holes in four different sheep. The sheep were anesthetized and the temperature elevations were measured with a thermocouple which was inserted in a previously drilled hole next to the planned drilling trajectory. Bone density distributions were extracted from pre-operative CT data by averaging Hounsfield values over the drill bit diameter. Post-operative [Formula: see text]CT data was used to verify the drilling accuracy of the trajectories. The comparison of measured and calculated temperatures shows a very good match for both heating and cooling phases. The average prediction error of the maximum temperature was less than 0.7 °C and the average root mean square error was approximately 0.5 °C. To analyze potential thermal damage, the model was used to calculate temperature profiles and cumulative equivalent minutes at 43 °C at a minimal distance to the facial nerve. For the selected drilling parameters, temperature elevation profiles and cumulative equivalent minutes suggest that thermal elevation of this minimally invasive cochlear implantation surgery may pose a risk to the facial nerve, especially in sclerotic or high density mastoid bones. Optimized drilling parameters need to be evaluated and the model could be used for future risk evaluation.

A Neuromonitoring Approach to Facial Nerve Preservation During Image-guided Robotic Cochlear Implantation.

HYPOTHESIS: A multielectrode probe in combination with an optimized stimulation protocol could provide sufficient sensitivity and specificity to act as an effective safety mechanism for preservation of the facial nerve in case of an unsafe drill distance during image-guided cochlear implantation. BACKGROUND: A minimally invasive cochlear implantation is enabled by image-guided and robotic-assisted drilling of an access tunnel to the middle ear cavity. The approach requires the drill to pass at distances below 1  mm from the facial nerve and thus safety mechanisms for protecting this critical structure are required. Neuromonitoring is currently used to determine facial nerve proximity in mastoidectomy but lacks sensitivity and specificity necessaries to effectively distinguish the close distance ranges experienced in the minimally invasive approach, possibly because of current shunting of uninsulated stimulating drilling tools in the drill tunnel and because of nonoptimized stimulation parameters. To this end, we propose an advanced neuromonitoring approach using varying levels of stimulation parameters together with an integrated bipolar and monopolar stimulating probe. MATERIALS AND METHODS: An in vivo study (sheep model) was conducted in which measurements at specifically planned and navigated lateral distances from the facial nerve were performed to determine if specific sets of stimulation parameters in combination with the proposed neuromonitoring system could reliably detect an imminent collision with the facial nerve. For the accurate positioning of the neuromonitoring probe, a dedicated robotic system for image-guided cochlear implantation was used and drilling accuracy was corrected on postoperative microcomputed tomographic images. RESULTS: From 29 trajectories analyzed in five different subjects, a correlation between stimulus threshold and drill-to-facial nerve distance was found in trajectories colliding with the facial nerve (distance <0.1  mm). The shortest pulse duration that provided the highest linear correlation between stimulation intensity and drill-to-facial nerve distance was 250  µs. Only at low stimulus intensity values (≤0.3  mA) and with the bipolar configurations of the probe did the neuromonitoring system enable sufficient lateral specificity (>95%) at distances to the facial nerve below 0.5  mm. However, reduction in stimulus threshold to 0.3  mA or lower resulted in a decrease of facial nerve distance detection range below 0.1  mm (>95% sensitivity). Subsequent histopathology follow-up of three representative cases where the neuromonitoring system could reliably detect a collision with the facial nerve (distance <0.1  mm) revealed either mild or inexistent damage to the nerve fascicles. CONCLUSION: Our findings suggest that although no general correlation between facial nerve distance and stimulation threshold existed, possibly because of variances in patient-specific anatomy, correlations at very close distances to the facial nerve and high levels of specificity would enable a binary response warning system to be developed using the proposed probe at low stimulation currents.

Manual Electrode Array Insertion Through a Robot-Assisted Minimal Invasive Cochleostomy: Feasibility and Comparison of Two Different Electrode Array Subtypes.

HYPOTHESIS: To evaluate the feasibility and the results of insertion of two types of electrode arrays in a robotically assisted surgical approach. BACKGROUND: Recent publications demonstrated that robot-assisted surgery allows the implantation of free-fitting electrode arrays through a cochleostomy drilled via a narrow bony tunnel (DCA). We investigated if electrode arrays from different manufacturers could be used with this approach. METHODS: Cone-beam CT imaging was performed on five-cadaveric heads after placement of fiducial screws. Relevant anatomical structures were segmented and the DCA trajectory, including the position of the cochleostomy, was defined to target the center of the scala tympani while reducing the risk of lesions to the facial nerve. Med-El Flex 28 and Cochlear CI422 electrodes were implanted on both sides, and their position was verified by cone-beam CT. Finally, temporal bones were dissected to assess the occurrence of damage to anatomical structures during DCA drilling. RESULTS: The cochleostomy site was directed in the scala tympani in 9 of 10 cases. The insertion of electrode arrays was successful in 19 of 20 attempts. No facial nerve damage was observed. The average difference between the planned and the postoperative trajectory was 0.17 ± 0.19 mm at the level of the facial nerve. The average depth of insertion was 305.5 ± 55.2 and 243 ± 32.1 degrees with Med-El and Cochlear arrays, respectively. CONCLUSIONS: Robot-assisted surgery is a reliable tool to allow cochlear implantation through a cochleostomy. Technical solutions must be developed to improve the electrode array insertion using this approach.

Facial nerve image enhancement from CBCT using supervised learning technique.

Facial nerve segmentation plays an important role in surgical planning of cochlear implantation. Clinically available CBCT images are used for surgical planning. However, its relatively low resolution renders the identification of the facial nerve difficult. In this work, we present a supervised learning approach to enhance facial nerve image information from CBCT. A supervised learning approach based on multi-output random forest was employed to learn the mapping between CBCT and micro-CT images. Evaluation was performed qualitatively and quantitatively by using the predicted image as input for a previously published dedicated facial nerve segmentation, and cochlear implantation surgical planning software, OtoPlan. Results show the potential of the proposed approach to improve facial nerve image quality as imaged by CBCT and to leverage its segmentation using OtoPlan.

An image-guided robot system for direct cochlear access.

The aim of direct cochlear access (DCA) is to replace the standard mastoidectomy with a small diameter tunnel from the lateral bone surface to the cochlea for electrode array insertion. In contrast to previous attempts, the approach described in this work not only achieves an unprecedented high accuracy, but also contains several safety sub-systems. This paper provides a brief description of the system components, and summarizes accuracy results using the system in a cadaver model over the past two years.

Feasibility of using EMG for early detection of the facial nerve during robotic direct cochlear access.

HYPOTHESIS: Facial nerve monitoring can be used synchronous with a high-precision robotic tool as a functional warning to prevent of a collision of the drill bit with the facial nerve during direct cochlear access (DCA). BACKGROUND: Minimally invasive direct cochlear access (DCA) aims to eliminate the need for a mastoidectomy by drilling a small tunnel through the facial recess to the cochlea with the aid of stereotactic tool guidance. Because the procedure is performed in a blind manner, structures such as the facial nerve are at risk. Neuromonitoring is a commonly used tool to help surgeons identify the facial nerve (FN) during routine surgical procedures in the mastoid. Recently, neuromonitoring technology was integrated into a commercially available drill system enabling real-time monitoring of the FN. The objective of this study was to determine if this drilling system could be used to warn of an impending collision with the FN during robot-assisted DCA. MATERIALS AND METHODS: The sheep was chosen as a suitable model for this study because of its similarity to the human ear anatomy. The same surgical workflow applicable to human patients was performed in the animal model. Bone screws, serving as reference fiducials, were placed in the skull near the ear canal. The sheep head was imaged using a computed tomographic scanner and segmentation of FN, mastoid, and other relevant structures as well as planning of drilling trajectories was carried out using a dedicated software tool. During the actual procedure, a surgical drill system was connected to a nerve monitor and guided by a custom built robot system. As the planned trajectories were drilled, stimulation and EMG response signals were recorded. A postoperative analysis was achieved after each surgery to determine the actual drilled positions. RESULTS: Using the calibrated pose synchronized with the EMG signals, the precise relationship between distance to FN and EMG with 3 different stimulation intensities could be determined for 11 different tunnels drilled in 3 different subjects. CONCLUSION: From the results, it was determined that the current implementation of the neuromonitoring system lacks sensitivity and repeatability necessary to be used as a warning device in robotic DCA. We hypothesize that this is primarily because of the stimulation pattern achieved using a noninsulated drill as a stimulating probe. Further work is necessary to determine whether specific changes to the design can improve the sensitivity and specificity.

Surgical planning tool for robotically assisted hearing aid implantation.

PURPOSE: For the facilitation of minimally invasive robotically performed direct cochlea access (DCA) procedure, a surgical planning tool which enables the surgeon to define landmarks for patient-to-image registration, identify the necessary anatomical structures and define a safe DCA trajectory using patient image data (typically computed tomography (CT) or cone beam CT) is required. To this end, a dedicated end-to-end software planning system for the planning of DCA procedures that addresses current deficiencies has been developed. METHODS: Efficient and robust anatomical segmentation is achieved through the implementation of semiautomatic algorithms; high-accuracy patient-to-image registration is achieved via an automated model-based fiducial detection algorithm and functionality for the interactive definition of a safe drilling trajectory based on case-specific drill positioning uncertainty calculations was developed. RESULTS: The accuracy and safety of the presented software tool were validated during the conduction of eight DCA procedures performed on cadaver heads. The plan for each ear was completed in less than 20 min, and no damage to vital structures occurred during the procedures. The integrated fiducial detection functionality enabled final positioning accuracies of 0.15 ± 0.08 mm. CONCLUSIONS: Results of this study demonstrated that the proposed software system could aid in the safe planning of a DCA tunnel within an acceptable time.

In vitro accuracy evaluation of image-guided robot system for direct cochlear access.

HYPOTHESIS: A previously developed image-guided robot system can safely drill a tunnel from the lateral mastoid surface, through the facial recess, to the middle ear, as a viable alternative to conventional mastoidectomy for cochlear electrode insertion. BACKGROUND: Direct cochlear access (DCA) provides a minimally invasive tunnel from the lateral surface of the mastoid through the facial recess to the middle ear for cochlear electrode insertion. A safe and effective tunnel drilled through the narrow facial recess requires a highly accurate image-guided surgical system. Previous attempts have relied on patient-specific templates and robotic systems to guide drilling tools. In this study, we report on improvements made to an image-guided surgical robot system developed specifically for this purpose and the resulting accuracy achieved in vitro. MATERIALS AND METHODS: The proposed image-guided robotic DCA procedure was carried out bilaterally on 4 whole head cadaver specimens. Specimens were implanted with titanium fiducial markers and imaged with cone-beam CT. A preoperative plan was created using a custom software package wherein relevant anatomical structures of the facial recess were segmented, and a drill trajectory targeting the round window was defined. Patient-to-image registration was performed with the custom robot system to reference the preoperative plan, and the DCA tunnel was drilled in 3 stages with progressively longer drill bits. The position of the drilled tunnel was defined as a line fitted to a point cloud of the segmented tunnel using principle component analysis (PCA function in MatLab). The accuracy of the DCA was then assessed by coregistering preoperative and postoperative image data and measuring the deviation of the drilled tunnel from the plan. The final step of electrode insertion was also performed through the DCA tunnel after manual removal of the promontory through the external auditory canal. RESULTS: Drilling error was defined as the lateral deviation of the tool in the plane perpendicular to the drill axis (excluding depth error). Errors of 0.08 ± 0.05 mm and 0.15 ± 0.08 mm were measured on the lateral mastoid surface and at the target on the round window, respectively (n =8). Full electrode insertion was possible for 7 cases. In 1 case, the electrode was partially inserted with 1 contact pair external to the cochlea. CONCLUSION: The purpose-built robot system was able to perform a safe and reliable DCA for cochlear implantation. The workflow implemented in this study mimics the envisioned clinical procedure showing the feasibility of future clinical implementation.

Augmented environments for the targeting of hepatic lesions during image-guided robotic liver surgery.

BACKGROUND: Stereotactic navigation technology can enhance guidance during surgery and enable the precise reproduction of planned surgical strategies. Currently, specific systems (such as the CAS-One system) are available for instrument guidance in open liver surgery. This study aims to evaluate the implementation of such a system for the targeting of hepatic tumors during robotic liver surgery. MATERIAL AND METHODS: Optical tracking references were attached to one of the robotic instruments and to the robotic endoscopic camera. After instrument and video calibration and patient-to-image registration, a virtual model of the tracked instrument and the available three-dimensional images of the liver were displayed directly within the robotic console, superimposed onto the endoscopic video image. An additional superimposed targeting viewer allowed for the visualization of the target tumor, relative to the tip of the instrument, for an assessment of the distance between the tumor and the tool for the realization of safe resection margins. RESULTS: Two cirrhotic patients underwent robotic navigated atypical hepatic resections for hepatocellular carcinoma. The augmented endoscopic view allowed for the definition of an accurate resection margin around the tumor. The overlay of reconstructed three-dimensional models was also used during parenchymal transection for the identification of vascular and biliary structures. Operative times were 240 min in the first case and 300 min in the second. There were no intraoperative complications. CONCLUSIONS: The da Vinci Surgical System provided an excellent platform for image-guided liver surgery with a stable optic and instrumentation. Robotic image guidance might improve the surgeon's orientation during the operation and increase accuracy in tumor resection. Further developments of this technological combination are needed to deal with organ deformation during surgery.

High-accuracy patient-to-image registration for the facilitation of image-guided robotic microsurgery on the head.

Image-guided microsurgery requires accuracies an order of magnitude higher than today's navigation systems provide. A critical step toward the achievement of such low-error requirements is a highly accurate and verified patient-to-image registration. With the aim of reducing target registration error to a level that would facilitate the use of image-guided robotic microsurgery on the rigid anatomy of the head, we have developed a semiautomatic fiducial detection technique. Automatic force-controlled localization of fiducials on the patient is achieved through the implementation of a robotic-controlled tactile search within the head of a standard surgical screw. Precise detection of the corresponding fiducials in the image data is realized using an automated model-based matching algorithm on high-resolution, isometric cone beam CT images. Verification of the registration technique on phantoms demonstrated that through the elimination of user variability, clinically relevant target registration errors of approximately 0.1 mm could be achieved.

A portable image overlay projection device for computer-aided open liver surgery.

Image overlay projection is a form of augmented reality that allows surgeons to view underlying anatomical structures directly on the patient surface. It improves intuitiveness of computer-aided surgery by removing the need for sight diversion between the patient and a display screen and has been reported to assist in 3-D understanding of anatomical structures and the identification of target and critical structures. Challenges in the development of image overlay technologies for surgery remain in the projection setup. Calibration, patient registration, view direction, and projection obstruction remain unsolved limitations to image overlay techniques. In this paper, we propose a novel, portable, and handheld-navigated image overlay device based on miniature laser projection technology that allows images of 3-D patient-specific models to be projected directly onto the organ surface intraoperatively without the need for intrusive hardware around the surgical site. The device can be integrated into a navigation system, thereby exploiting existing patient registration and model generation solutions. The position of the device is tracked by the navigation system's position sensor and used to project geometrically correct images from any position within the workspace of the navigation system. The projector was calibrated using modified camera calibration techniques and images for projection are rendered using a virtual camera defined by the projectors extrinsic parameters. Verification of the device's projection accuracy concluded a mean projection error of 1.3 mm. Visibility testing of the projection performed on pig liver tissue found the device suitable for the display of anatomical structures on the organ surface. The feasibility of use within the surgical workflow was assessed during open liver surgery. We show that the device could be quickly and unobtrusively deployed within the sterile environment.