HoloLens in Breast Reconstruction: What Is the Future?

SUMMARY: Autologous breast reconstruction using the deep inferior epigastric perforator flap has been established as the standard for perforator-based free-flap breast reconstruction. This technique relies on the surgeon's ability to identify the patient's relevant abdominal vasculature to facilitate accurate dissection, optimize surgical outcomes, and minimize morbidity. A technique is described in which the authors incorporate augmented reality using HoloLens technology in their surgical planning to identify epigastric arteries and perforators. This technology allows the surgeon to superimpose computed tomography angiography images directly onto the patient, facilitating an in vivo appreciation of underlying anatomy before incision and dissection. This allows real-time surgical planning, increasing the value and tangibility of preoperative computed tomography angiography with the potential to enhance the accuracy and efficiency of the operative technique. Although the authors did not use the HoloLens technology to make clinical decisions, they provide evidence of its accuracy and ease of use, offering a proof of concept. The potential of this technology is demonstrated, and the authors encourage future application in free-flap breast reconstruction and beyond.

A remote access mixed reality teaching ward round.

BACKGROUND: Heterogeneous access to clinical learning opportunities and inconsistency in teaching is a common source of dissatisfaction among medical students. This was exacerbated during the COVID-19 pandemic, with limited exposure to patients for clinical teaching. METHODS: We conducted a proof-of-concept study at a London teaching hospital using mixed reality (MR) technology (HoloLens2™) to deliver a remote access teaching ward round. RESULTS: Students unanimously agreed that use of this technology was enjoyable and provided teaching that was otherwise inaccessible. The majority of participants gave positive feedback on the MR (holographic) content used (n = 8 out of 11) and agreed they could interact with and have their questions answered by the clinician leading the ward round (n = 9). Quantitative and free text feedback from students, patients and faculty members demonstrated that this is a feasible, acceptable and effective method for delivery of clinical education. DISCUSSION: We have used this technology in a novel way to transform the delivery of medical education and enable consistent access to high-quality teaching. This can now be integrated across the curriculum and will include remote access to specialist clinics and surgery. A library of bespoke MR educational resources will be created for future generations of medical students and doctors to use on an international scale.

Visual behaviour in robotic surgery-Demonstrating the validity of the simulated environment.

BACKGROUND: Eye metrics provide insight into surgical behaviour allowing differentiation of performance, however have not been used in robotic surgery. This study explores eye metrics of robotic surgeons in training in simulated and real tissue environments. METHODS: Following the Fundamentals of Robotic Surgery (FRS), training curriculum novice robotic surgeons were trained to expert-derived benchmark proficiency using real tissue on the da Vinci Si and the da Vinci skills simulator (dVSS) simulator. Surgeons eye metrics were recorded using eye-tracking glasses when both "novice" and "proficient" in both environments. Performance was assessed using Global Evaluative Assessment of Robotic skills (GEARS) and numeric psychomotor test score (NPMTS) scores. RESULTS: Significant (P ≤ .05) correlations were seen between pupil size, rate of change and entropy, and associated GEARS/NPMTS in "novice" and "proficient" surgeons. Only number of blinks per minute was significantly different between pupilometrics in the simulated and real tissue environments. CONCLUSIONS: This study illustrates the value of eye tracking as an objective physiological tool in the robotic setting. Pupilometrics significantly correlate with established assessment methods and could be incorporated into robotic surgery assessments.

Patient-Specific Simulation of Pneumoperitoneum for Laparoscopic Surgical Planning.

Gas insufflation in laparoscopy deforms the abdomen and stretches the overlying skin. This limits the use of surgical image-guidance technologies and challenges the appropriate placement of trocars, which influences the operative ease and potential quality of laparoscopic surgery. This work describes the development of a platform that simulates pneumoperitoneum in a patient-specific manner, using preoperative CT scans as input data. This aims to provide a more realistic representation of the intraoperative scenario and guide trocar positioning to optimize the ergonomics of laparoscopic instrumentation. The simulation was developed by generating 3D reconstructions of insufflated and deflated porcine CT scans and simulating an artificial pneumoperitoneum on the deflated model. Simulation parameters were optimized by minimizing the discrepancy between the simulated pneumoperitoneum and the ground truth model extracted from insufflated porcine scans. Insufflation modeling in humans was investigated by correlating the simulation's output to real post-insufflation measurements obtained from patients in theatre. The simulation returned an average error of 7.26 mm and 10.5 mm in the most and least accurate datasets respectively. In context of the initial discrepancy without simulation (23.8 mm and 19.6 mm), the methods proposed here provide a significantly improved picture of the intraoperative scenario. The framework was also demonstrated capable of simulating pneumoperitoneum in humans. This study proposes a method for realistically simulating pneumoperitoneum to achieve optimal ergonomics during laparoscopy. Although further studies to validate the simulation in humans are needed, there is the opportunity to provide a more realistic, interactive simulation platform for future image-guided minimally invasive surgery.

Evaluating the impact of image guidance in the surgical setting: a systematic review.

BACKGROUND: Image guidance has been clinically available for over a period of 20 years. Although research increasingly has a translational emphasis, overall the clinical uptake of image guidance systems in surgery remains low. The objective of this review was to establish the metrics used to report on the impact of surgical image guidance systems used in a clinical setting. METHODS: A systematic review of the literature was carried out on all relevant publications between January 2000 and April 2016. Ovid MEDLINE and Embase databases were searched using a title strategy. Reported outcome metrics were grouped into clinically relevant domains and subsequent sub-categories for analysis. RESULTS: In total, 232 publications were eligible for inclusion. Analysis showed that clinical outcomes and system interaction were consistently reported. However, metrics focusing on surgeon, patient and economic impact were reported less often. No increase in the quality of reporting was observed during the study time period, associated with study design, or when the clinical setting involved a surgical specialty that had been using image guidance for longer. CONCLUSIONS: Publications reporting on the clinical use of image guidance systems are evaluating traditional surgical outcomes and neglecting important human and economic factors, which are pertinent to the uptake, diffusion and sustainability of image-guided surgery. A framework is proposed to assist researchers in providing comprehensive evaluation metrics, which should also be considered in the design phase. Use of these would help demonstrate the impact in the clinical setting leading to increased clinical integration of image guidance systems.

Subject-specific modelling of pneumoperitoneum: model implementation, validation and human feasibility assessment.

PURPOSE: The aim of this study is to propose a model that simulates patient-specific anatomical changes resulting from pneumoperitoneum, using preoperative data as input. The framework can assist the surgeon through a real-time visualisation and interaction with the model. Such could further facilitate surgical planning preoperatively, by defining a surgical strategy, and intraoperatively to estimate port positions. METHODS: The biomechanical model that simulates pneumoperitoneum was implemented within the GPU-accelerated NVIDIA FleX position-based dynamics framework. Datasets of multiple porcine subjects before and after abdominal insufflation were used to generate, calibrate and validate the model. The feasibility of modelling pneumoperitoneum in human subjects was assessed by comparing distances between specific landmarks from a patient abdominal wall, to the same landmark measurements on the simulated model. RESULTS: The calibration of simulation parameters resulted in a successful estimation of an optimal set parameters. A correspondence between the simulation pressure parameter and the experimental insufflation pressure was determined. The simulation of pneumoperitoneum in a porcine subject resulted in a mean Hausdorff distance error of 5-6 mm. Feasibility of modelling pneumoperitoneum in humans was successfully demonstrated. CONCLUSION: Simulation of pneumoperitoneum provides an accurate subject-specific 3D model of the inflated abdomen, which is a more realistic representation of the intraoperative scenario when compared to preoperative imaging alone. The simulation results in a stable and interactive framework that performs in real time, and supports patient-specific data, which can assist in surgical planning.

Perfect Registration Leads to Imperfect Performance: A Randomized Trial of Multimodal Intraoperative Image Guidance.

OBJECTIVE: To compare surgical safety and efficiency of 2 image guidance modalities, perfect augmented reality (AR) and side-by-side unregistered image guidance (IG), against a no guidance control (NG), when performing a simulated laparoscopic cholecystectomy (LC). BACKGROUND: Image guidance using AR offers the potential to improve understanding of subsurface anatomy, with positive ramifications for surgical safety and efficiency. No intra-abdominal study has demonstrated any advantage for the technology. Perfect AR cannot be provided in the operative setting in a patient; however, it can be generated in the simulated setting. METHODS: Thirty-six experienced surgeons performed a baseline LC using the LapMentor simulator before randomization to 1 of 3 study arms: AR, IG, or NG. Each performed 3 further LC. Safety and efficiency-related simulator metrics, and task workload (SURG-TLX) were collected. RESULTS: The IG group had a shorter total instrument path length and fewer movements than NG and AR groups. Both IG and NG took a significantly shorter time than AR to complete dissection of Calot triangle. Use of IG and AR resulted in significantly fewer perforations and serious complications than the NG group. IG had significantly fewer perforations and serious complications than the AR group. Compared with IG, AR guidance was found to be significantly more distracting. CONCLUSION: Side-by-side unregistered image guidance (IG) improved safety and surgical efficiency in a simulated setting when compared with AR or NG. IG provides a more tangible opportunity for integrating image guidance into existing surgical workflow as well as delivering the safety and efficiency benefits desired.

Intraoperative ultrasound for improved 3D tumour reconstruction in robot-assisted surgery: An evaluation of feedback modalities.

BACKGROUND: Intraoperative ultrasound scanning induces deformation on the tissue in the absence of a feedback modality, which results in a 3D tumour reconstruction that is not directly representative of real anatomy. METHODS: A biomechanical model with different feedback modalities (haptic, visual, or auditory) was implemented in a simulation environment. A user study with 20 clinicians was performed to assess which modality resulted in the 3D tumour volume reconstruction that most resembled the reference configuration from the respective computed tomography (CT) scans. RESULTS: Integrating a feedback modality significantly improved the scanning performance across all participants and data sets. The optimal feedback modality to adopt varied depending on the evaluation. Nonetheless, using guidance with feedback is always preferred compared with none. CONCLUSIONS: The results demonstrated the urgency to integrate a feedback modality framework into clinical practice, to ensure an improved scanning performance. Furthermore, this framework enabled an evaluation that cannot be performed in vivo.

Current and Future Use of Radiological Images in the Management of Gynecological Malignancies - A Survey of Practice in the UK.

BACKGROUND/AIM: Radiology provides increasingly accurate and complex information. Understanding the clinicians' interpretation of scans could improve surgical planning, decision-making; informed training and development of augmented imaging. This was a survey exploring the interpretation of imaging by clinicians and its use in operative preparation and prediction. MATERIALS AND METHODS: The survey was open for two-months and circulated online to British Gynaecological Cancer society members. RESULTS: Seventy-three (19%) members completed the survey. Respondents had a confidence level of 51% in their ability to interpret computed tomography (CT) and/or magnetic resonance imaging (MRI) images independently. Preoperative imaging was commonly used to plan operations, predict complications and complete resection. Images were reviewed for primary (96.3%)/interval (92.6%) ovarian debulking, but less so for vulvectomy (45%). Scan (79.6%) and multidisciplinary team meeting (MDT) (66.6%) reports were used more often than scan images (50%) for operative planning. Amount and pattern of disease on scan were the most important factors predicting operating time. CONCLUSION: Imaging influences the surgeon's planning, however respondents lack confidence. Training of clinicians in radiological interpretation needs to improve. Augmented image interfaces could facilitate this.

Transoral Robotic Surgery: Image Guidance and Augmented Reality.

Transoral robotic surgery (TORS) is being used increasingly as an alternative to conventional open procedures, as its minimally invasive approach results in significant benefits for the patient. This includes faster recovery times and better functional outcomes, less postoperative pain and morbidity associated with open access, shorter operative times, and a reduction in the risk of infection. However, the confined working spaces and sensory deficit, due to a lack of haptic sensation, and proximity to critical vasculature and other vitally important anatomical structures bestow significant challenges on the surgeon. Recent advances in the field of surgical image guidance show promise in helping address these issues. This overview considers a number of important recent developments, all of which exploit imaging technologies that, to a lesser or greater extent, are integrated with the surgical robot platform. In its most advanced form, this combination of technologies results in a fusion of visual information such that the operative field is combined with navigational cues and representations of key anatomical structures through augmented reality registration and overlay. In terms of registration accuracy, image guidance as applied to TORS procedures is an exciting proposition, since the regions of interest (e.g., the tongue base, oropharynx) are typically adjacent to and enclosed by rigid anatomy. Clinical indications and overall effectiveness are discussed together with the limitations of the current technologies. Where appropriate, directions for future work are identified.

Through the HoloLens™ looking glass: augmented reality for extremity reconstruction surgery using 3D vascular models with perforating vessels.

Precision and planning are key to reconstructive surgery. Augmented reality (AR) can bring the information within preoperative computed tomography angiography (CTA) imaging to life, allowing the surgeon to 'see through' the patient's skin and appreciate the underlying anatomy without making a single incision. This work has demonstrated that AR can assist the accurate identification, dissection and execution of vascular pedunculated flaps during reconstructive surgery. Separate volumes of osseous, vascular, skin, soft tissue structures and relevant vascular perforators were delineated from preoperative CTA scans to generate three-dimensional images using two complementary segmentation software packages. These were converted to polygonal models and rendered by means of a custom application within the HoloLens™ stereo head-mounted display. Intraoperatively, the models were registered manually to their respective subjects by the operating surgeon using a combination of tracked hand gestures and voice commands; AR was used to aid navigation and accurate dissection. Identification of the subsurface location of vascular perforators through AR overlay was compared to the positions obtained by audible Doppler ultrasound. Through a preliminary HoloLens-assisted case series, the operating surgeon was able to demonstrate precise and efficient localisation of perforating vessels.

Follow the light: projector-based augmented reality intracorporeal system for laparoscopic surgery.

A projector-based augmented reality intracorporeal system (PARIS) is presented that includes a miniature tracked projector, tracked marker, and laparoscopic ultrasound (LUS) transducer. PARIS was developed to improve the efficacy and safety of laparoscopic partial nephrectomy (LPN). In particular, it has been demonstrated to effectively assist in the identification of tumor boundaries during surgery and to improve the surgeon's understanding of the underlying anatomy. PARIS achieves this by displaying the orthographic projection of the cancerous tumor on the kidney's surface. The performance of PARIS was evaluated in a user study with two surgeons who performed 32 simulated robot-assisted partial nephrectomies. They performed 16 simulated partial nephrectomies with PARIS for guidance and 16 simulated partial nephrectomies with only an LUS transducer for guidance. With PARIS, there was a significant reduction [30% ([Formula: see text])] in the amount of healthy tissue excised and a trend toward a more accurate dissection around the tumor and more negative margins. The combined point tracking and reprojection root-mean-square error of PARIS was 0.8 mm. PARIS' proven ability to improve key metrics of LPN surgery and qualitative feedback from surgeons about PARIS supports the hypothesis that it is an effective surgical navigation tool.

Intra-operative ultrasound-based augmented reality guidance for laparoscopic surgery.

In laparoscopic surgery, the surgeon must operate with a limited field of view and reduced depth perception. This makes spatial understanding of critical structures difficult, such as an endophytic tumour in a partial nephrectomy. Such tumours yield a high complication rate of 47%, and excising them increases the risk of cutting into the kidney's collecting system. To overcome these challenges, an augmented reality guidance system is proposed. Using intra-operative ultrasound, a single navigation aid, and surgical instrument tracking, four augmentations of guidance information are provided during tumour excision. Qualitative and quantitative system benefits are measured in simulated robot-assisted partial nephrectomies. Robot-to-camera calibration achieved a total registration error of 1.0 ± 0.4 mm while the total system error is 2.5 ± 0.5 mm. The system significantly reduced healthy tissue excised from an average (±standard deviation) of 30.6 ± 5.5 to 17.5 ± 2.4 cm3 (p < 0.05) and reduced the depth from the tumor underside to cut from an average (±standard deviation) of 10.2 ± 4.1 to 3.3 ± 2.3 mm (p < 0.05). Further evaluation is required in vivo, but the system has promising potential to reduce the amount of healthy parenchymal tissue excised.

A Novel High-Throughput Screening Platform Reveals an Optimized Cytokine Formulation for Human Hematopoietic Progenitor Cell Expansion.

The main limitations of hematopoietic cord blood (CB) transplantation, viz, low cell dosage and delayed reconstitution, can be overcome by ex vivo expansion. CB expansion under conventional culture causes rapid cell differentiation and depletion of hematopoietic stem and progenitor cells (HSPCs) responsible for engraftment. In this study, we use combinatorial cell culture technology (CombiCult®) to identify medium formulations that promote CD133+ CB HSPC proliferation while maintaining their phenotypic characteristics. We employed second-generation CombiCult screens that use electrospraying technology to encapsulate CB cells in alginate beads. Our results suggest that not only the combination but also the order of addition of individual components has a profound influence on expansion of specific HSPC populations. Top protocols identified by the CombiCult screen were used to culture human CD133+ CB HSPCs on nanofiber scaffolds and validate the expansion of the phenotypically defined CD34+CD38lo/-CD45RA-CD90+CD49f+ population of hematopoietic stem cells and their differentiation into defined progeny.

Soft tissue deformation for surgical simulation: a position-based dynamics approach.

PURPOSE: To assist the rehearsal and planning of robot-assisted partial nephrectomy, a real-time simulation platform is presented that allows surgeons to visualise and interact with rapidly constructed patient-specific biomechanical models of the anatomical regions of interest. Coupled to a framework for volumetric deformation, the platform furthermore simulates intracorporeal 2D ultrasound image acquisition, using preoperative imaging as the data source. This not only facilitates the planning of optimal transducer trajectories and viewpoints, but can also act as a validation context for manually operated freehand 3D acquisitions and reconstructions. METHODS: The simulation platform was implemented within the GPU-accelerated NVIDIA FleX position-based dynamics framework. In order to validate the model and determine material properties and other simulation parameter values, a porcine kidney with embedded fiducial beads was CT-scanned and segmented. Acquisitions for the rest position and three different levels of probe-induced deformation were collected. Optimal values of the cluster stiffness coefficients were determined for a range of different particle radii, where the objective function comprised the mean distance error between real and simulated fiducial positions over the sequence of deformations. RESULTS: The mean fiducial error at each deformation stage was found to be compatible with the level of ultrasound probe calibration error typically observed in clinical practice. Furthermore, the simulation exhibited unconditional stability on account of its use of clustered shape-matching constraints. CONCLUSIONS: A novel position-based dynamics implementation of soft tissue deformation has been shown to facilitate several desirable simulation characteristics: real-time performance, unconditional stability, rapid model construction enabling patient-specific behaviour and accuracy with respect to reference CT images.

Using preoperative imaging for intraoperative guidance: a case of mistaken identity.

BACKGROUND: Surgical image guidance systems to date have tended to rely on reconstructions of preoperative datasets. This paper assesses the accuracy of these reconstructions to establish whether they are appropriate for use in image guidance platforms. METHODS: Nine raters (two experts in image interpretation and preparation, three in image interpretation, and four in neither interpretation nor preparation) were asked to perform a segmentation of ten renal tumours (four cystic and six solid tumours). These segmentations were compared with a gold standard consensus segmentation generated using a previously validated algorithm. RESULTS: Average sensitivity and positive predictive value (PPV) were 0.902 and 0.891, respectively. When assessing for variability between raters, significant differences were seen in the PPV, sensitivity and incursions and excursions from consensus tumour boundary. CONCLUSIONS: This paper has demonstrated that the interpretation required for the segmentation of preoperative imaging of renal tumours introduces significant inconsistency and inaccuracy. Copyright © 2015 John Wiley & Sons, Ltd.

Evaluating a Novel 3D Stereoscopic Visual Display for Transanal Endoscopic Surgery: A Randomized Controlled Crossover Study.

OBJECTIVE: To compare surgical performance with transanal endoscopic surgery (TES) using a novel 3-dimensional (3D) stereoscopic viewer against the current modalities of a 3D stereoendoscope, 3D, and 2-dimensional (2D) high-definition monitors. BACKGROUND: TES is accepted as the primary treatment for selected rectal tumors. Current TES systems offer a 2D monitor, or 3D image, viewed directly via a stereoendoscope, necessitating an uncomfortable operating position. To address this and provide a platform for future image augmentation, a 3D stereoscopic display was created. METHODS: Forty participants, of mixed experience level, completed a simulated TES task using 4 visual displays (novel stereoscopic viewer and currently utilized stereoendoscope, 3D, and 2D high-definition monitors) in a randomly allocated order. Primary outcome measures were: time taken, path length, and accuracy. Secondary outcomes were: task workload and participant questionnaire results. RESULTS: Median time taken and path length were significantly shorter for the novel viewer versus 2D and 3D, and not significantly different to the traditional stereoendoscope. Significant differences were found in accuracy, task workload, and questionnaire assessment in favor of the novel viewer, as compared to all 3 modalities. CONCLUSIONS: This novel 3D stereoscopic viewer allows surgical performance in TES equivalent to that achieved using the current stereoendoscope and superior to standard 2D and 3D displays, but with lower physical and mental demands for the surgeon. Participants expressed a preference for this system, ranking it more highly on a questionnaire. Clinical translation of this work has begun with the novel viewer being used in 5 TES patients.

Comparative effectiveness and safety of image guidance systems in surgery: a preclinical randomised study.

BACKGROUND: Over the past decade image guidance systems have been widely adopted in specialties such as neurosurgery and otorhinolaryngology. Nonetheless, the evidence supporting the use of image guidance systems in surgery remains limited. New augmented reality systems offer the possibility of enhanced operating room workflow compared with existing triplanar image displays, but recent studies have highlighted several concerns, particularly the risk of inattentional blindness and impaired depth perception. The aim of this study was to compare simultaneously the effectiveness and safety of various image guidance systems against standard surgery. METHODS: In this preclinical randomised study design 50 novice surgeons were allocated to no image guidance, triplanar display, always-on solid overlay, always-on wire mesh overlay, or on-demand inverse realism overlay. Each participant was asked to identify a basilar tip aneurysm in a validated model head. The primary outcomes were time to task completion, and tool path length. The secondary outcomes were recognition of an unexpected finding (a surgical clip) and subjective depth perception (using a Likert scale). FINDINGS: Surgeons' time to task completion and tool path length were significantly lower in groups using any form of image guidance than in groups with no image guidance (p<0·001 and p=0·003, respectively). The tool path distance was also lower in groups using augmented reality than in those using triplanar display (p=0·010). Always-on solid overlay resulted in the greatest inattentional blindness (20% recognition of unexpected finding by all surgeons). Wire mesh and on-demand overlays mitigated but did not negate inattentional blindness, and were comparable with triplanar display (40% recognition of unexpected finding in all groups). Wire mesh and inverse realism overlays also resulted in better subjective depth perception than always-on solid overlay (p=0·031 and p=0·008, respectively). INTERPRETATION: This study suggests that new augmented reality platforms incorporating always-on wire mesh and on-demand inverse realism might improve surgical performance, at least in novice surgeons. All image display modalities, including existing triplanar display, carry a risk of inattentional blindness. FUNDING: Wellcome Trust.

Robust ultrasound probe tracking: initial clinical experiences during robot-assisted partial nephrectomy.

PURPOSE: In order to assist in the identification of renal vasculature and tumour boundaries in robot-assisted partial nephrectomy, robust ultrasound probe calibration and tracking methods are introduced. Contemporaneous image guidance during these crucial stages of the procedure should ultimately lead to improved safety and quality of outcome for the patient, through reduced positive margin rates, segmental clamping, shorter ischaemic times and nephron-sparing resection. METHODS: Small KeyDot markers with circular dot patterns are attached to a miniature pickup ultrasound probe. Generic probe calibration is superseded by a more robust scheme based on a sequence of physical transducer measurements. Motion prediction combined with a reduced region-of-interest in the endoscopic video feed facilitates real-time tracking and registration performance at full HD resolutions. RESULTS: Quantitative analysis confirms that circular dot patterns result in an improved translational and rotational working envelope, in comparison with the previous chessboard pattern implementation. Furthermore, increased robustness is observed with respect to prevailing illumination levels and out-of-focus images due to relatively small endoscopic depths of field. CONCLUSION: Circular dot patterns should be employed in this context as they result in improved performance and robustness. This facilitates clinical usage and interpretation of the combined video and ultrasound overlay. The efficacy of the overall system is demonstrated in the first human clinical case.

Pico Lantern: Surface reconstruction and augmented reality in laparoscopic surgery using a pick-up laser projector.

The Pico Lantern is a miniature projector developed for structured light surface reconstruction, augmented reality and guidance in laparoscopic surgery. During surgery it will be dropped into the patient and picked up by a laparoscopic tool. While inside the patient it projects a known coded pattern and images onto the surface of the tissue. The Pico Lantern is visually tracked in the laparoscope's field of view for the purpose of stereo triangulation between it and the laparoscope. In this paper, the first application is surface reconstruction. Using a stereo laparoscope and an untracked Pico Lantern, the absolute error for surface reconstruction for a plane, cylinder and ex vivo kidney, is 2.0 mm, 3.0 mm and 5.6 mm, respectively. Using a mono laparoscope and a tracked Pico Lantern for the same plane, cylinder and kidney the absolute error is 1.4 mm, 1.5 mm and 1.5 mm, respectively. These results confirm the benefit of the wider baseline produced by tracking the Pico Lantern. Virtual viewpoint images are generated from the kidney surface data and an in vivo proof-of-concept porcine trial is reported. Surface reconstruction of the neck of a volunteer shows that the pulsatile motion of the tissue overlying a major blood vessel can be detected and displayed in vivo. Future work will integrate the Pico Lantern into standard and robot-assisted laparoscopic surgery.