Iatrogenic inner ear dehiscence associated with lateral skull base surgery: a systematic analysis of drilling injuries and their causal factors.

PURPOSE: Drilling injuries of the inner ear are an underreported complication of lateral skull base (LSB) surgery. Inner ear breaches can cause hearing loss, vestibular dysfunction, and third window phenomenon. This study aims to elucidate primary factors causing iatrogenic inner ear dehiscences (IED) in 9 patients who presented to a tertiary care center with postoperative symptoms of IED following LSB surgery for vestibular schwannoma, endolymphatic sac tumor, Meniere's disease, paraganglioma jugulare, and vagal schwannoma. METHODS: Utilizing 3D Slicer image processing software, geometric and volumetric analysis was applied to both preoperative and postoperative imaging to identify causal factors iatrogenic inner ear breaches. Segmentation analyses, craniotomy analyses, and drilling trajectory analyses were performed. Cases of retrosigmoid approaches for vestibular schwannoma resection were compared to matched controls. RESULTS: Excessive lateral drilling and breach of a single inner ear structure occurred in 3 cases undergoing transjugular (n=2) and transmastoid (n=1) approaches. Inadequate drilling trajectory breaching ≥1 inner ear structure occurred in 6 cases undergoing retrosigmoid (n=4), transmastoid (n=1), and middle cranial fossa approaches (n=1). In retrosigmoid approaches the 2-cm visualization window and craniotomy limits did not provide drilling angles to the entire tumor without causing IED in comparison to matched controls. CONCLUSIONS: Inappropriate drill depth, errant lateral drilling, inadequate drill trajectory, or a combination of these led to iatrogenic IED. Image-based segmentation, individualized 3D anatomical model generation, and geometric and volumetric analyses can optimize operative plans and possibly reduce inner ear breaches from lateral skull base surgery.

Computer-aided heterogeneity analysis in breast MR imaging assessment of ductal carcinoma in situ: Correlating histologic grade and receptor status.

PURPOSE: To identify breast MR imaging biomarkers to predict histologic grade and receptor status of ductal carcinoma in situ (DCIS). MATERIALS AND METHODS: Informed consent was waived in this Health Insurance Portability and Accountability Act-compliant Institutional Review Board-approved study. Case inclusion was conducted from 7332 consecutive breast MR studies from January 1, 2009, to December 31, 2012. Excluding studies with benign diagnoses, studies without visible abnormal enhancement, and pathology containing invasive disease yielded 55 MR-imaged pathology-proven DCIS seen on 54 studies. Twenty-eight studies (52%) were performed at 1.5 Tesla (T); 26 (48%) at 3T. Regions-of-interest representing DCIS were segmented for precontrast, first and fourth postcontrast, and subtracted first and fourth postcontrast images on the open-source three-dimensional (3D) Slicer software. Fifty-seven metrics of each DCIS were obtained, including distribution statistics, shape, morphology, Renyi dimensions, geometrical measure, and texture, using the 3D Slicer HeterogeneityCAD module. Statistical correlation of heterogeneity metrics with DCIS grade and receptor status was performed using univariate Mann-Whitney test. RESULTS: Twenty-four of the 55 DCIS (44%) were high nuclear grade (HNG); 44 (80%) were estrogen receptor (ER) positive. Human epidermal growth factor receptor-2 (HER2) was amplified in 10/55 (18%), nonamplified in 34/55 (62%), unknown/equivocal in 8/55 (15%). Surface area-to-volume ratio showed significant difference (P < 0.05) between HNG and non-HNG DCIS. No metric differentiated ER status (0.113 < p ≤ 1.000). Seventeen metrics showed significant differences between HER2-positive and HER2-negative DCIS (0.016 < P < 0.050). CONCLUSION: Quantitative heterogeneity analysis of DCIS suggests the presence of MR imaging biomarkers in classifying DCIS grade and HER2 status. Validation with larger samples and prospective studies is needed to translate these results into clinical applications. LEVEL OF EVIDENCE: 3 Technical Efficacy: Stage 3 J. Magn. Reson. Imaging 2017;46:1748-1759.

Computer-Aided Diagnosis of Ground-Glass Opacity Nodules Using Open-Source Software for Quantifying Tumor Heterogeneity.

OBJECTIVE: The purposes of this study are to develop quantitative imaging biomarkers obtained from high-resolution CTs for classifying ground-glass nodules (GGNs) into atypical adenomatous hyperplasia (AAH), adenocarcinoma in situ (AIS), minimally invasive adenocarcinoma (MIA), and invasive adenocarcinoma (IAC); to evaluate the utility of contrast enhancement for differential diagnosis; and to develop and validate a support vector machine (SVM) to predict the GGN type. MATERIALS AND METHODS: The heterogeneity of 248 GGNs was quantified using custom software. Statistical analysis with a univariate Kruskal-Wallis test was performed to evaluate metrics for significant differences among the four GGN groups. The heterogeneity metrics were used to train a SVM to learn and predict the lesion type. RESULTS: Fifty of 57 and 51 of 57 heterogeneity metrics showed statistically significant differences among the four GGN groups on unenhanced and contrast-enhanced CT scans, respectively. The SVM predicted lesion type with greater accuracy than did three expert radiologists. The accuracy of classifying the GGNs into the four groups on the basis of the SVM algorithm was 70.9%, whereas the accuracy of the radiologists was 39.6%. The accuracy of SVM in classifying the AIS and MIA nodules was 73.1%, and the accuracy of the radiologists was 35.7%. For indolent versus invasive lesions, the accuracy of the SVM was 88.1%, and the accuracy of the radiologists was 60.8%. We found that contrast enhancement does not significantly improve the differential diagnosis of GGNs. CONCLUSION: Compared with the GGN classification done by the three radiologists, the SVM trained regarding all the heterogeneity metrics showed significantly higher accuracy in classifying the lesions into the four groups, differentiating between AIS and MIA and between indolent and invasive lesions. Contrast enhancement did not improve the differential diagnosis of GGNs.

Feasibility of Intraoperative Breast MRI and the Role of Prone Versus Supine Positioning in Surgical Planning for Breast-Conserving Surgery.

We assessed the feasibility of supine intraoperative MRI (iMRI) during breast-conserving surgery (BCS), enrolling 15 patients in our phase I trial between 2012 and 2014. Patients received diagnostic prone MRI, BCS, pre-excisional supine iMRI, and postexcisional supine iMRI. Feasibility was assessed based on safety, sterility, duration, and image-quality. Twelve patients completed the study; mean duration = 114 minutes; all images were adequate; no complications, safety, or sterility issues were encountered. Substantial tumor-associated changes occurred (mean displacement = 67.7 mm, prone-supine metric, n = 7). We have demonstrated iMRI feasibility for BCS and have identified potential limitations of prone breast MRI that may impact surgical planning.

Intraoperative Supine Breast MR Imaging to Quantify Tumor Deformation and Detection of Residual Breast Cancer: Preliminary Results.

Purpose To use intraoperative supine magnetic resonance (MR) imaging to quantify breast tumor deformation and displacement secondary to the change in patient positioning from imaging (prone) to surgery (supine) and to evaluate residual tumor immediately after breast-conserving surgery (BCS). Materials and Methods Fifteen women gave informed written consent to participate in this prospective HIPAA-compliant, institutional review board-approved study between April 2012 and November 2014. Twelve patients underwent lumpectomy and postsurgical intraoperative supine MR imaging. Six of 12 patients underwent both pre- and postsurgical supine MR imaging. Geometric, structural, and heterogeneity metrics of the cancer and distances of the tumor from the nipple, chest wall, and skin were computed. Mean and standard deviations of the changes in volume, surface area, compactness, spherical disproportion, sphericity, and distances from key landmarks were computed from tumor models. Imaging duration was recorded. Results The mean differences in tumor deformation metrics between prone and supine imaging were as follows: volume, 23.8% (range, -30% to 103.95%); surface area, 6.5% (range, -13.24% to 63%); compactness, 16.2% (range, -23% to 47.3%); sphericity, 6.8% (range, -9.10% to 20.78%); and decrease in spherical disproportion, -11.3% (range, -60.81% to 76.95%). All tumors were closer to the chest wall on supine images than on prone images. No evidence of residual tumor was seen on MR images obtained after the procedures. Mean duration of pre- and postoperative supine MR imaging was 25 minutes (range, 18.4-31.6 minutes) and 19 minutes (range, 15.1-22.9 minutes), respectively. Conclusion Intraoperative supine breast MR imaging, when performed in conjunction with standard prone breast MR imaging, enables quantification of breast tumor deformation and displacement secondary to changes in patient positioning from standard imaging (prone) to surgery (supine) and may help clinicians evaluate for residual tumor immediately after BCS. © RSNA, 2016 Online supplemental material is available for this article.

Multimodal imaging for improved diagnosis and treatment of cancers.

The authors review methods for image-guided diagnosis and therapy that increase precision in the detection, characterization, and localization of many forms of cancer to achieve optimal target definition and complete resection or ablation. A new model of translational, clinical, image-guided therapy research is presented, and the Advanced Multimodality Image-Guided Operating (AMIGO) suite is described. AMIGO was conceived and designed to allow for the full integration of imaging in cancer diagnosis and treatment. Examples are drawn from over 500 procedures performed on brain, neck, spine, thorax (breast, lung), and pelvis (prostate and gynecologic) areas and are used to describe how they address some of the many challenges of treating brain, prostate, and lung tumors. Cancer 2015;121:817-827. © 2014 American Cancer Society.

Image-guided video assisted thoracoscopic surgery (iVATS) - phase I-II clinical trial.

PURPOSE: To facilitate localization and resection of small lung nodules, we developed a prospective clinical trial (ClinicalTrials.gov number NCT01847209) for a novel surgical approach which combines placement of fiducials using intra-operative C-arm computed tomography (CT) guidance with standard thoracoscopic resection technique using image-guided video-assisted thoracoscopic surgery (iVATS). METHODS: Pretrial training was performed in a porcine model using C-arm CT and needle guidance software. Methodology and workflow for iVATS was developed, and a multi-modality team was trained. A prospective phase I-II clinical trial was initiated with the goal of recruiting eligible patients with small peripheral pulmonary nodules. Intra-operative C-arm CT scan was utilized for guidance of percutaneous marking with two T-bars (Kimberly-Clark, Roswell, GA) followed by VATS resection of the tumor. RESULTS: Twenty-five patients were enrolled; 23 underwent iVATS, one withdrew, and one lesion resolved. Size of lesions were: 0.6-1.8 cm, mean = 1.3 ± 0.38 cm.. All 23 patients underwent complete resection of their lesions. CT imaging of the resected specimens confirmed the removal of the T-bars and the nodule. Average and total procedure radiation dose was in the acceptable low range (median = 1501 µGy*m(2), range 665-16,326). There were no deaths, and all patients were discharged from the hospital (median length of stay = 4 days, range 2-12). Three patients had postoperative complications: one prolonged air-leak, one pneumonia, and one ileus. CONCLUSIONS: A successful and safe step-wise process has been established for iVATS, combining intra-operative C-arm CT scanning and thoracoscopic surgery in a hybrid operating room.

Inner Ear Breaches from Vestibular Schwannoma Surgery: Revisiting the Incidence of Otologic Injury from Retrosigmoid and Middle Cranial Fossa Approaches.

OBJECTIVE: To assess the rate of iatrogenic injury to the inner ear in vestibular schwannoma resections. STUDY DESIGN: Retrospective case review. SETTING: Multiple academic tertiary care hospitals. PATIENTS: Patients who underwent retrosigmoid or middle cranial fossa approaches for vestibular schwannoma resection between 1993 and 2015. INTERVENTION: Diagnostic with therapeutic implications. MAIN OUTCOME MEASURE: Drilling breach of the inner ear as confirmed by operative note or postoperative computed tomography (CT). RESULTS: 21.5% of patients undergoing either retrosigmoid or middle fossa approaches to the internal auditory canal were identified with a breach of the vestibulocochlear system. Because of the lack of postoperative CT imaging in this cohort, this is likely an underestimation of the true incidence of inner ear breaches. Of all postoperative CT scans reviewed, 51.8% had an inner ear breach. As there may be bias in patients undergoing postoperative CT, a middle figure based on sensitivity analyses estimates the incidence of inner ear breaches from lateral skull base surgery to be 34.7%. CONCLUSIONS: A high percentage of vestibular schwannoma surgeries via retrosigmoid and middle cranial fossa approaches result in drilling breaches of the inner ear. This study reinforces the value of preoperative image analysis for determining risk of inner ear breaches during vestibular schwannoma surgery and the importance of acquiring CT studies postoperatively to evaluate the integrity of the inner ear.

Separable Tendon-Driven Robotic Manipulator with a Long, Flexible, Passive Proximal Section.

This work tackles practical issues which arise when using a tendon-driven robotic manipulator (TDRM) with a long, flexible, passive proximal section in medical applications. Tendon-driven devices are preferred in medicine for their improved outcomes via minimally invasive procedures, but TDRMs come with unique challenges such as sterilization and reuse, simultaneous control of tendons, hysteresis in the tendon-sheath mechanism, and unmodeled effects of the proximal section shape. A separable TDRM which overcomes difficulties in actuation and sterilization is introduced, in which the body containing the electronics is reusable and the remainder is disposable. An open-loop redundant controller which resolves the redundancy in the kinematics is developed. Simple linear hysteresis compensation and re-tension compensation based on the physical properties of the device are proposed. The controller and compensation methods are evaluated on a testbed for a straight proximal section, a curved proximal section at various static angles, and a proximal section which dynamically changes angles; and overall, distal tip error was reduced.

EMDQ: Removal of Image Feature Mismatches in Real-Time.

This paper proposes a novel method for removing image feature mismatches in real-time that can handle both rigid and smooth deforming environments. Image distortion, parallax and object deformation may cause the pixel coordinates of feature matches to have non-rigid deformations, which cannot be represented using a single analytical rigid transformation. To solve this problem, we propose an algorithm based on the re-weighting and 1-point RANSAC strategy (R1P-RNSC), which operates under the assumption that a non-rigid deformation can be approximately represented by multiple rigid transformations. R1P-RNSC is fast but suffers from the drawback that local smoothing information cannot be considered, thus limiting its accuracy. To solve this problem, we propose a non-parametric algorithm based on the expectation-maximization algorithm and the dual quaternion-based representation (EMDQ). EMDQ generates dense and smooth deformation fields by interpolating among the feature matches, simultaneously removing mismatches that are inconsistent with the deformation field. It relies on the rigid transformations obtained by R1P-RNSC to improve its accuracy. The experimental results demonstrate that EMDQ has superior accuracy compared to other state-of-the-art mismatch removal methods. The ability to build correspondences for all image pixels using the dense deformation field is another contribution of this paper.

Evaluation of colonoscopy technical skill levels by use of an objective kinematic-based system.

BACKGROUND: Colonoscopy requires training and experience to ensure accuracy and safety. Currently, no objective, validated process exists to determine when an endoscopist has attained technical competence. Kinematics data describing movements of laparoscopic instruments have been used in surgical skill assessment to define expert surgical technique. We have developed a novel system to record kinematics data during colonoscopy and quantitatively assess colonoscopist performance. OBJECTIVE: To use kinematic analysis of colonoscopy to quantitatively assess endoscopic technical performance. DESIGN: Prospective cohort study. SETTING: Tertiary-care academic medical center. POPULATION: This study involved physicians who perform colonoscopy. INTERVENTION: Application of a kinematics data collection system to colonoscopy evaluation. MAIN OUTCOME MEASUREMENTS: Kinematics data, validated task load assessment instrument, and technical difficulty visual analog scale. RESULTS: All 13 participants completed the colonoscopy to the terminal ileum on the standard colon model. Attending physicians reached the terminal ileum quicker than fellows (median time, 150.19 seconds vs 299.86 seconds; p<.01) with reduced path lengths for all 4 sensors, decreased flex (1.75 m vs 3.14 m; P=.03), smaller tip angulation, reduced absolute roll, and lower curvature of the endoscope. With performance of attending physicians serving as the expert reference standard, the mean kinematic score increased by 19.89 for each decrease in postgraduate year (P<.01). Overall, fellows experienced greater mental, physical, and temporal demand than did attending physicians. LIMITATION: Small cohort size. CONCLUSION: Kinematic data and score calculation appear useful in the evaluation of colonoscopy technical skill levels. The kinematic score appears to consistently vary by year of training. Because this assessment is nonsubjective, it may be an improvement over current methods for determination of competence. Ongoing studies are establishing benchmarks and characteristic profiles of skill groups based on kinematics data.

Real-Time Nonrigid Mosaicking of Laparoscopy Images.

The ability to extend the field of view of laparoscopy images can help the surgeons to obtain a better understanding of the anatomical context. However, due to tissue deformation, complex camera motion and significant three-dimensional (3D) anatomical surface, image pixels may have non-rigid deformation and traditional mosaicking methods cannot work robustly for laparoscopy images in real-time. To solve this problem, a novel two-dimensional (2D) non-rigid simultaneous localization and mapping (SLAM) system is proposed in this paper, which is able to compensate for the deformation of pixels and perform image mosaicking in real-time. The key algorithm of this 2D non-rigid SLAM system is the expectation maximization and dual quaternion (EMDQ) algorithm, which can generate smooth and dense deformation field from sparse and noisy image feature matches in real-time. An uncertainty-based loop closing method has been proposed to reduce the accumulative errors. To achieve real-time performance, both CPU and GPU parallel computation technologies are used for dense mosaicking of all pixels. Experimental results on in vivo and synthetic data demonstrate the feasibility and accuracy of our non-rigid mosaicking method.

EMDQ-SLAM: Real-time High-resolution Reconstruction of Soft Tissue Surface from Stereo Laparoscopy Videos.

We propose a novel stereo laparoscopy video-based non-rigid SLAM method called EMDQ-SLAM, which can incrementally reconstruct thee-dimensional (3D) models of soft tissue surfaces in real-time and preserve high-resolution color textures. EMDQ-SLAM uses the expectation maximization and dual quaternion (EMDQ) algorithm combined with SURF features to track the camera motion and estimate tissue deformation between video frames. To overcome the problem of accumulative errors over time, we have integrated a g2o-based graph optimization method that combines the EMDQ mismatch removal and as-rigid-as-possible (ARAP) smoothing methods. Finally, the multi-band blending (MBB) algorithm has been used to obtain high resolution color textures with real-time performance. Experimental results demonstrate that our method outperforms two state-of-the-art non-rigid SLAM methods: MISSLAM and DefSLAM. Quantitative evaluation shows an average error in the range of 0.8-2.2 mm for different cases.

F3RNet: full-resolution residual registration network for deformable image registration.

PURPOSE: Deformable image registration (DIR) is essential for many image-guided therapies. Recently, deep learning approaches have gained substantial popularity and success in DIR. Most deep learning approaches use the so-called mono-stream high-to-low, low-to-high network structure and can achieve satisfactory overall registration results. However, accurate alignments for some severely deformed local regions, which are crucial for pinpointing surgical targets, are often overlooked. Consequently, these approaches are not sensitive to some hard-to-align regions, e.g., intra-patient registration of deformed liver lobes. METHODS: We propose a novel unsupervised registration network, namely full-resolution residual registration network (F3RNet), for deformable registration of severely deformed organs. The proposed method combines two parallel processing streams in a residual learning fashion. One stream takes advantage of the full-resolution information that facilitates accurate voxel-level registration. The other stream learns the deep multi-scale residual representations to obtain robust recognition. We also factorize the 3D convolution to reduce the training parameters and enhance network efficiency. RESULTS: We validate the proposed method on a clinically acquired intra-patient abdominal CT-MRI dataset and a public inspiratory and expiratory thorax CT dataset. Experiments on both multimodal and unimodal registration demonstrate promising results compared to state-of-the-art approaches. CONCLUSION: By combining the high-resolution information and multi-scale representations in a highly interactive residual learning fashion, the proposed F3RNet can achieve accurate overall and local registration. The run time for registering a pair of images is less than 3 s using a GPU. In future works, we will investigate how to cost-effectively process high-resolution information and fuse multi-scale representations.

3D deep learning based classification of pulmonary ground glass opacity nodules with automatic segmentation.

Classifying ground-glass lung nodules (GGNs) into atypical adenomatous hyperplasia (AAH), adenocarcinoma in situ (AIS), minimally invasive adenocarcinoma (MIA), and invasive adenocarcinoma (IAC) on diagnostic CT images is important to evaluate the therapy options for lung cancer patients. In this paper, we propose a joint deep learning model where the segmentation can better facilitate the classification of pulmonary GGNs. Based on our observation that masking the nodule to train the model results in better lesion classification, we propose to build a cascade architecture with both segmentation and classification networks. The segmentation model works as a trainable preprocessing module to provide the classification-guided 'attention' weight map to the raw CT data to achieve better diagnosis performance. We evaluate our proposed model and compare with other baseline models for 4 clinically significant nodule classification tasks, defined by a combination of pathology types, using 4 classification metrics: Accuracy, Average F1 Score, Matthews Correlation Coefficient (MCC), and Area Under the Receiver Operating Characteristic Curve (AUC). Experimental results show that the proposed method outperforms other baseline models on all the diagnostic classification tasks.

A novel dual-network architecture for mixed-supervised medical image segmentation.

In medical image segmentation tasks, deep learning-based models usually require densely and precisely annotated datasets to train, which are time-consuming and expensive to prepare. One possible solution is to train with the mixed-supervised dataset, where only a part of data is densely annotated with segmentation map and the rest is annotated with some weak form, such as bounding box. In this paper, we propose a novel network architecture called Mixed-Supervised Dual-Network (MSDN), which consists of two separate networks for the segmentation and detection tasks respectively, and a series of connection modules between the layers of the two networks. These connection modules are used to extract and transfer useful information from the detection task to help the segmentation task. We exploit a variant of a recently designed technique called 'Squeeze and Excitation' in the connection module to boost the information transfer between the two tasks. Compared with existing model with shared backbone and multiple branches, our model has flexible and trainable feature sharing fashion and thus is more effective and stable. We conduct experiments on 4 medical image segmentation datasets, and experiment results show that the proposed MSDN model outperforms multiple baselines.

Mixed-Supervised Dual-Network for Medical Image Segmentation.

Deep learning based medical image segmentation models usually require large datasets with high-quality dense segmentations to train, which are very time-consuming and expensive to prepare. One way to tackle this difficulty is using the mixed-supervised learning framework, where only a part of data is densely annotated with segmentation label and the rest is weakly labeled with bounding boxes. The model is trained jointly in a multi-task learning setting. In this paper, we propose Mixed-Supervised Dual-Network (MSDN), a novel architecture which consists of two separate networks for the detection and segmentation tasks respectively, and a series of connection modules between the layers of the two networks. These connection modules are used to transfer useful information from the auxiliary detection task to help the segmentation task. We propose to use a recent technique called 'Squeeze and Excitation' in the connection module to boost the transfer. We conduct experiments on two medical image segmentation datasets. The proposed MSDN model outperforms multiple baselines.

A Novel Mixed Reality Navigation System for Laparoscopy Surgery.

OBJECTIVE: To design and validate a novel mixed reality head-mounted display for intraoperative surgical navigation. DESIGN: A mixed reality navigation for laparoscopic surgery (MRNLS) system using a head mounted display (HMD) was developed to integrate the displays from a laparoscope, navigation system, and diagnostic imaging to provide context-specific information to the surgeon. Further, an immersive auditory feedback was also provided to the user. Sixteen surgeons were recruited to quantify the differential improvement in performance based on the mode of guidance provided to the user (laparoscopic navigation with CT guidance (LN-CT) versus mixed reality navigation for laparoscopic surgery (MRNLS)). The users performed three tasks: (1) standard peg transfer, (2) radiolabeled peg identification and transfer, and (3) radiolabeled peg identification and transfer through sensitive wire structures. RESULTS: For the more complex task of peg identification and transfer, significant improvements were observed in time to completion, kinematics such as mean velocity, and task load index subscales of mental demand and effort when using the MRNLS (p < 0.05) compared to the current standard of LN-CT. For the final task of peg identification and transfer through sensitive structures, time taken to complete the task and frustration were significantly lower for MRNLS compared to the LN-CT approach. CONCLUSIONS: A novel mixed reality navigation for laparoscopic surgery (MRNLS) has been designed and validated. The ergonomics of laparoscopic procedures could be improved while minimizing the necessity of additional monitors in the operating room.

Kalman filter-based EM-optical sensor fusion for needle deflection estimation.

PURPOSE: In many clinical procedures such as cryoablation that involves needle insertion, accurate placement of the needle's tip at the desired target is the major issue for optimizing the treatment and minimizing damage to the neighboring anatomy. However, due to the interaction force between the needle and tissue, considerable error in intraoperative tracking of the needle tip can be observed as needle deflects. METHODS: In this paper, measurements data from an optical sensor at the needle base and a magnetic resonance (MR) gradient field-driven electromagnetic (EM) sensor placed 10 cm from the needle tip are used within a model-integrated Kalman filter-based sensor fusion scheme. Bending model-based estimations and EM-based direct estimation are used as the measurement vectors in the Kalman filter, thus establishing an online estimation approach. RESULTS: Static tip bending experiments show that the fusion method can reduce the mean error of the tip position estimation from 29.23 mm of the optical sensor-based approach to 3.15 mm of the fusion-based approach and from 39.96 to 6.90 mm, at the MRI isocenter and the MRI entrance, respectively. CONCLUSION: This work established a novel sensor fusion scheme that incorporates model information, which enables real-time tracking of needle deflection with MRI compatibility, in a free-hand operating setup.