AI-Dentify: deep learning for proximal caries detection on bitewing x-ray - HUNT4 Oral Health Study.

BACKGROUND: Dental caries diagnosis requires the manual inspection of diagnostic bitewing images of the patient, followed by a visual inspection and probing of the identified dental pieces with potential lesions. Yet the use of artificial intelligence, and in particular deep-learning, has the potential to aid in the diagnosis by providing a quick and informative analysis of the bitewing images. METHODS: A dataset of 13,887 bitewings from the HUNT4 Oral Health Study were annotated individually by six different experts, and used to train three different object detection deep-learning architectures: RetinaNet (ResNet50), YOLOv5 (M size), and EfficientDet (D0 and D1 sizes). A consensus dataset of 197 images, annotated jointly by the same six dental clinicians, was used for evaluation. A five-fold cross validation scheme was used to evaluate the performance of the AI models. RESULTS: The trained models show an increase in average precision and F1-score, and decrease of false negative rate, with respect to the dental clinicians. When compared against the dental clinicians, the YOLOv5 model shows the largest improvement, reporting 0.647 mean average precision, 0.548 mean F1-score, and 0.149 mean false negative rate. Whereas the best annotators on each of these metrics reported 0.299, 0.495, and 0.164 respectively. CONCLUSION: Deep-learning models have shown the potential to assist dental professionals in the diagnosis of caries. Yet, the task remains challenging due to the artifacts natural to the bitewing images.

A novel clip-on device for electromagnetic tracking in endobronchial ultrasound bronchoscopy.

INTRODUCTION: The established method for assessment of mediastinal and hilar lymph nodes is endobronchial ultrasound bronchoscopy (EBUS) with needle aspirations. Previously, we presented an electromagnetic navigation platform for this purpose. There were several issues with the permanent electromagnetic tracking (EMT) sensor attachment on the tip of the experimental EBUS bronchoscope. The purpose was to develop a device for on-site attachment of the EMT sensor. MATERIAL AND METHODS: A clip-on EMT sensor attachment device was 3D-printed in Ultem™ and attached to an EBUS bronchoscope. A specially designed ultrasound probe calibration adapter was developed for on-site and quick probe calibration. Navigation accuracy was studied using a wire cross water phantom and clinical feasibility was tested in a healthy volunteer. RESULTS: The device attached to the EBUS bronchoscope increased its diameter from 6.9 mm to 9.5 mm. Average preclinical navigation accuracy was 3.9 mm after adapter calibration. The maneuvering of the bronchoscope examining a healthy volunteer was adequate without harming the respiratory epithelium, and the device stayed firmly attached. CONCLUSION: Development, calibration and testing of a clip-on EMT sensor attachment device for EBUS bronchoscopy was successfully demonstrated. Acceptable accuracy results were obtained, and the device is ready to be tested in patient studies.

Can effective pedagogy be ensured in minimally invasive surgery e-learning?

INTRODUCTION: Effectiveness of e-learning diminishes without the support of a pedagogical model to guide its use. In minimally invasive surgery (MIS), this has been reported as a limitation when technology is used to deliver contents without a sound pedagogical background. MATERIAL AND METHODS: We describe how a generic pedagogical model, the 3D pedagogy framework, can be used for setting learning outcomes and activities in e-learning platforms focused on MIS cognitive skills. A demonstrator course on Nissen fundoplication was developed following the model step-by-step in the MISTELA learning platform. Course design was informed by Kolb's Experiential learning model. Content validation was performed by 13 MIS experts. RESULTS: Ten experts agreed on the suitability of content structuring done according to the pedagogical model. All experts agreed that the course provides means to assess the intended learning outcomes. CONCLUSIONS: This work showcases how a general-purpose e-learning framework can be accommodated to the needs of MIS training without limiting the course designers' pedagogical approach. Key advances for its success include: (1) proving the validity of the model in the wider scope of MIS skills and (2) raising awareness amongst stakeholders on the need of developing training plans with explicit, rather than assumed, pedagogical foundations. Abbreviations: MIS: minimally invasive surgery; TEL: technology enhanced learning.

Pulmonologist evaluation on new CT visualization for guidance to lung lesions during bronchoscopy.

OBJECTIVE: Endoluminal visualization in virtual and video bronchoscopy lacks information about the surrounding structures, and the traditional 2 D axial, coronal and sagittal CT views can be difficult to interpret. To address this challenge, we previously introduced a novel visualization technique, Anchored to Centerline Curved Surface, for navigated bronchoscopy. The current study compares the ACCuSurf to the standard ACS CT views as planning and guiding tools in a phantom study. MATERIAL AND METHODS: Bronchoscope operators navigated in physical phantom guided by virtual realistic image data constructed by fusion of CT dataset of phantom and anonymized patient CT data. We marked four different target positions within the virtual image data and gave 12 pulmonologists the task to navigate, with either ACCuSurf or ACS as guidance, to the corresponding targets in the physical phantom. RESULTS: Using ACCuSurf reduced the planning time and increased the grade of successful navigation significantly compared to ACS. CONCLUSION: The phantom setup with virtual patient image data proved realistic according to the pulmonologists. ACCuSurf proved superior to ACS regarding planning time and navigation success grading. Improvements on visualisation or display techniques may consequently improve both planning and navigated bronchoscopy and thus contribute to more precise lung diagnostics.

Peripheral tumour targeting using open-source virtual bronchoscopy with electromagnetic tracking: a multi-user pre-clinical study.

Objectives: The goal was to demonstrate the utility of open-source tracking and visualisation tools in the targeting of lung cancer.Material and methods: The study demonstrates the first deployment of the Anser electromagnetic (EM) tracking system with the CustusX image-guided interventional research platform to navigate using an endobronchial catheter to injected tumour targets. Live animal investigations validated the deployment and targeting of peripheral tumour models using an innovative tumour marking routine.Results: Novel tumour model deployment was successfully achieved at all eight target sites across two live animal investigations without pneumothorax. Virtual bronchoscopy with tracking successfully guided the tracked catheter to 2-12 mm from the target tumour site. Deployment of a novel marker was achieved at all eight sites providing a reliable measure of targeting accuracy. Targeting accuracy within 10 mm was achieved in 7/8 sites and in all cases, the virtual target distance at marker deployment was within the range subsequently measured with x-ray.Conclusions: Endobronchial targeting of peripheral airway targets is feasible using existing open-source technology. Notwithstanding the shortcomings of current commercial platforms, technological improvements in EM tracking and registration accuracy fostered by open-source technology may provide the impetus for widespread clinical uptake of electromagnetic navigation in bronchoscopy.

A new visualization method for navigated bronchoscopy.

OBJECTIVE: In flexible endoscopy techniques, such as bronchoscopy, there is often a challenge visualizing the path from start to target based on preoperative data and accessing these during the procedure. An example of this is visualizing only the inside of central airways in bronchoscopy. Virtual bronchoscopy (VB) does not meet the pulmonologist's need to detect, define and sample the frequent targets outside the bronchial wall. Our aim was to develop and study a new visualization technique for navigated bronchoscopy. MATERIAL AND METHODS: We extracted the shortest possible path from the top of the trachea to the target along the airway centerline and a corresponding auxiliary route in the opposite lung. A surface structure between the centerlines was developed and displayed. The new technique was tested on non-selective CT data from eight patients using artificial lung targets. RESULTS: The new display technique anchored to centerline curved surface (ACCuSurf) made it easy to detect and interpret anatomical features, targets and neighboring anatomy outside the airways, in all eight patients. CONCLUSIONS: ACCuSurf can simplify planning and performing navigated bronchoscopy, meets the challenge of improving orientation and register the direction of the moving endoscope, thus creating an optimal visualization for navigated bronchoscopy.

Psychomotor skills assessment by motion analysis in minimally invasive surgery on an animal organ.

BACKGROUND: A high level of psychomotor skills is required to perform minimally invasive surgery (MIS) safely. To be able to measure these skills is important in the assessment of surgeons, as it enables constructive feedback during training. The aim of this study was to test the validity of an objective and automatic assessment method using motion analysis during a laparoscopic procedure on an animal organ. MATERIAL AND METHODS: Experienced surgeons in laparoscopy (experts) and medical students (novices) performed a cholecystectomy on a porcine liver box model. The motions of the surgical tools were acquired and analyzed by 11 different motion-related metrics, i.e., a total of 19 metrics as eight of them were measured separately for each hand. We identified for which of the metrics the experts outperformed the novices. RESULTS: In total, two experts and 28 novices were included. The experts achieved significantly better results for 13 of the 19 instrument motion metrics. CONCLUSIONS: Expert performance is characterized by a low time to complete the cholecystectomy, high bimanual dexterity (instrument coordination), a limited amount of movement and low measurement of motion smoothness of the dissection instrument, and relatively high usage of the grasper to optimize tissue positioning for dissection.

Lack of transfer of skills after virtual reality simulator training with haptic feedback.

BACKGROUND AND OBJECTIVE: Virtual reality (VR) simulators enrich surgical training and offer training possibilities outside of the operating room (OR). In this study, we created a criterion-based training program on a VR simulator with haptic feedback and tested it by comparing the performances of a simulator group against a control group. MATERIAL AND METHODS: Medical students with no experience in laparoscopy were randomly assigned to a simulator group or a control group. In the simulator group, the candidates trained until they reached predefined criteria on the LapSim® VR simulator (Surgical Science AB, Göteborg, Sweden) with haptic feedback (XitactTM IHP, Mentice AB, Göteborg, Sweden). All candidates performed a cholecystectomy on a porcine organ model in a box trainer (the clinical setting). The performances were video rated by two surgeons blinded to subject training status. RESULTS: In total, 30 students performed the cholecystectomy and had their videos rated (N = 16 simulator group, N = 14 control group). The control group achieved better video rating scores than the simulator group (p < .05). CONCLUSIONS: The criterion-based training program did not transfer skills to the clinical setting. Poor mechanical performance of the simulated haptic feedback is believed to have resulted in a negative training effect.

Workflow and intervention times of MR-guided focused ultrasound - Predicting the impact of new techniques.

Magnetic resonance guided focused ultrasound surgery (MRgFUS) has become an attractive, non-invasive treatment for benign and malignant tumours, and offers specific benefits for poorly accessible locations in the liver. However, the presence of the ribcage and the occurrence of liver motion due to respiration limit the applicability MRgFUS. Several techniques are being developed to address these issues or to decrease treatment times in other ways. However, the potential benefit of such improvements has not been quantified. In this research, the detailed workflow of current MRgFUS procedures was determined qualitatively and quantitatively by using observation studies on uterine MRgFUS interventions, and the bottlenecks in MRgFUS were identified. A validated simulation model based on discrete events simulation was developed to quantitatively predict the effect of new technological developments on the intervention duration of MRgFUS on the liver. During the observation studies, the duration and occurrence frequencies of all actions and decisions in the MRgFUS workflow were registered, as were the occurrence frequencies of motion detections and intervention halts. The observation results show that current MRgFUS uterine interventions take on average 213min. Organ motion was detected on average 2.9 times per intervention, of which on average 1.0 actually caused a need for rework. Nevertheless, these motion occurrences and the actions required to continue after their detection consumed on average 11% and up to 29% of the total intervention duration. The simulation results suggest that, depending on the motion occurrence frequency, the addition of new technology to automate currently manual MRgFUS tasks and motion compensation could potentially reduce the intervention durations by 98.4% (from 256h 5min to 4h 4min) in the case of 90% motion occurrence, and with 24% (from 5h 19min to 4h 2min) in the case of no motion. In conclusion, new tools were developed to predict how intervention durations will be affected by future workflow changes and by the introduction of new technology.

European Association of Endoscopic Surgeons (EAES) consensus statement on the use of robotics in general surgery.

Following an extensive literature search and a consensus conference with subject matter experts the following conclusions can be drawn: 1. Robotic surgery is still at its infancy, and there is a great potential in sophisticated electromechanical systems to perform complex surgical tasks when these systems evolve. 2. To date, in the vast majority of clinical settings, there is little or no advantage in using robotic systems in general surgery in terms of clinical outcome. Dedicated parameters should be addressed, and high quality research should focus on quality of care instead of routine parameters, where a clear advantage is not to be expected. 3. Preliminary data demonstrates that robotic system have a clinical benefit in performing complex procedures in confined spaces, especially in those that are located in unfavorable anatomical locations. 4. There is a severe lack of high quality data on robotic surgery, and there is a great need for rigorously controlled, unbiased clinical trials. These trials should be urged to address the cost-effectiveness issues as well. 5. Specific areas of research should include complex hepatobiliary surgery, surgery for gastric and esophageal cancer, revisional surgery in bariatric and upper GI surgery, surgery for large adrenal masses, and rectal surgery. All these fields show some potential for a true benefit of using current robotic systems. 6. Robotic surgery requires a specific set of skills, and needs to be trained using a dedicated, structured training program that addresses the specific knowledge, safety issues and skills essential to perform this type of surgery safely and with good outcomes. It is the responsibility of the corresponding professional organizations, not the industry, to define the training and credentialing of robotic basic skills and specific procedures. 7. Due to the special economic environment in which robotic surgery is currently employed special care should be taken in the decision making process when deciding on the purchase, use and training of robotic systems in general surgery. 8. Professional organizations in the sub-specialties of general surgery should review these statements and issue detailed, specialty-specific guidelines on the use of specific robotic surgery procedures in addition to outlining the advanced robotic surgery training required to safely perform such procedures.

An integrated model-based software for FUS in moving abdominal organs.

Focused ultrasound surgery (FUS) is a non-invasive method for tissue ablation that has the potential for complete and controlled local tumour destruction with minimal side effects. The treatment of abdominal organs such as the liver, however, requires particular technological support in order to enable a safe, efficient and effective treatment. As FUS is applied from outside the patient's body, suitable imaging methods, such as magnetic resonance imaging or diagnostic ultrasound, are needed to guide and track the procedure. To facilitate an efficient FUS procedure in the liver, the organ motion during breathing and the partial occlusion by the rib cage need to be taken into account in real time, demanding a continuous patient-specific adaptation of the treatment configuration. Modelling the patient's respiratory motion and combining this with tracking data improves the accuracy of motion predictions. Modelling and simulation of the FUS effects within the body allows the use of treatment planning and has the potential to be used within therapy to increase knowledge about the patient status. This article describes integrated model-based software for patient-specific modelling and prediction for FUS treatments of moving abdominal organs.

Liver deformation in an animal model due to pneumoperitoneum assessed by a vessel-based deformable registration.

PURPOSE: Surgical navigation based on preoperative images partly overcomes some of the drawbacks of minimally invasive interventions - reduction of free sight, lack of dexterity and tactile feedback. The usefulness of preoperative images is limited in laparoscopic liver surgery, as the liver shifts due to respiration, induction of pneumoperitoneum and surgical manipulation. In this study, we evaluated the shift and deformation in an animal liver caused by respiration and pneumopertioneum using intraoperative cone beam CT. MATERIAL AND METHODS: 3D cone beam CT scans were acquired with arterial contrast. The centerlines of the segmented vessels were extracted from the images taken at different respiration and pressure settings. A non-rigid registration method was used to measure the shift and deformation. The mean Euclidean distance between the annotated landmarks was used for evaluation. RESULTS: A shift and deformation of 44.6 mm on average was introduced due to the combined effect of respiration and pneumoperitoneum. On average 91% of the deformations caused by the respiration and pneumoperitoneum were recovered. CONCLUSION: The results can contribute to the use of intraoperative imaging to correct for anatomic shift so that preoperative data can be used with greater confidence and accuracy during guidance of laparoscopic liver procedures.

Correction: Teacher-student approach for lung tumor segmentation from mixed-supervised datasets.

[This corrects the article DOI: 10.1371/journal.pone.0266147.].

Accuracy of instrument tip position using fiber optic shape sensing for navigated bronchoscopy.

The purpose of this study was to evaluate the accuracy of a method for estimating the tip position of a fiber optic shape-sensing (FOSS) integrated instrument being inserted through a bronchoscope. A modified guidewire with a multicore optical fiber was inserted into the working channel of a custom-made catheter with three electromagnetic (EM) sensors. The displacement between the instruments was manually set, and a point-based method was applied to match the position of the EM sensors to corresponding points on the shape. The accuracy was evaluated in a realistic bronchial model. An additional EM sensor was used to sample the tip of the guidewire, and the absolute deviation between this position and the estimated tip position was calculated. For small displacements between the tip of the FOSS integrated tool and the catheter, the median deviation in estimated tip position was ≤5 mm. For larger displacements, deviations exceeding 10 mm were observed. The deviations increased when the shape sensor had sharp curvatures relative to more straight shapes. The method works well for clinically relevant displacements of a biopsy tool from the bronchoscope tip, and when the path to the lesion has limited curvatures. However, improvements must be made to our configuration before pursuing further clinical testing.

Perceiving haptic feedback in virtual reality simulators.

BACKGROUND: To improve patient safety, training of psychomotor laparoscopic skills is often done on virtual reality (VR) simulators outside the operating room. Haptic sensations have been found to influence psychomotor performance in laparoscopy. The emulation of haptic feedback is thus an important aspect of VR simulation. Some VR simulators try to simulate these sensations with handles equipped with haptic feedback. We conducted a survey on how laparoscopic surgeons perceive handles with and without haptic feedback. METHODS: Surgeons with different levels of experience in laparoscopy were asked to test two handles: Xitact IHP with haptic feedback and Xitact ITP without haptic feedback (Mentice AB, Gothenburg, Sweden), connected to the LapSim (Surgical Science AB, Sweden) VR simulator. They performed two tasks on the simulator before answering 12 questions regarding the two handles. The surgeons were not informed about the differences in the handles. RESULTS: A total of 85 % of the 20 surgeons who participated in the survey claimed that it is important that handles with haptic feedback feel realistic. Ninety percent of the surgeons preferred the handles without haptic feedback. The friction in the handles with haptic feedback was perceived to be as in reality (5 %) or too high (95 %). Regarding the handles without haptic feedback, the friction was perceived as in reality (45 %), too low (50 %), or too high (5 %). A total of 85 % of the surgeons thought that the handle with haptic feedback attempts to simulate the resistance offered by tissue to deformation. Ten percent thought that the handle succeeds in doing so. CONCLUSIONS: The surveyed surgeons believe that haptic feedback is an important feature on VR simulators; however, they preferred the handles without haptic feedback because they perceived the handles with haptic feedback to add additional friction, making them unrealistic and not mechanically transparent.

A study of psychomotor skills in minimally invasive surgery: what differentiates expert and nonexpert performance.

BACKGROUND: A high level of psychomotor skills is required to perform minimally invasive surgery (MIS) safely. To assure high quality of skills, it is important to be able to measure and assess these skills. For that, it is necessary to determine aspects that indicate the difference between performances at various levels of proficiency. Measurement and assessment of skills in MIS are best done in an automatic and objective way. The goal of this study was to investigate a set of nine motion-related metrics for their relevance to assess psychomotor skills in MIS during the performance of a labyrinth task. METHODS: Thirty-two surgeons and medical students were divided into three groups according to their level of experience in MIS; experts (>500 MIS procedures), intermediates (31-500 MIS), and novices (no experience in MIS). The participants performed the labyrinth task in the D-box Basic simulator (D-Box Medical, Lier, Norway). The task required bimanual maneuvering and threading a needle through a labyrinth of 10 holes. Nine motion-related metrics were used to assess the MIS skills of each participant. RESULTS: Experts (n = 7) and intermediates (n = 14) performed significantly better than the novices (n = 11) in terms of time and parameters measuring the amount of instrument movement. The experts had significantly better bimanual dexterity, which indicated that they made more simultaneous movements of the two instruments compared to the intermediates and novices. The experts also performed the task with a shorter instrument path length with the nondominant hand than the intermediates. CONCLUSIONS: The surgeon's performance in MIS can be distinguished from a novice by metrics such as time and path length. An experienced surgeon in MIS can be differentiated from a less experienced one by the higher ability to control the instrument in the nondominant hand and the higher degree of simultaneous (coordinated) movements of the two instruments.

Limitations of haptic feedback devices on construct validity of the LapSim® virtual reality simulator.

BACKGROUND: Surgeons performing laparoscopy need a high degree of psychomotor skills, which can be trained and assessed on virtual reality (VR) simulators. VR simulators simulate the surgical environment and assess psychomotor skills according to predefined parameters. This study aimed to validate a proficiency-based training setup that consisted of two tasks with predefined threshold values and handles with haptic feedback on the LapSim(®) VR simulator. The two tasks have been found to have construct validity in previous studies using handles without haptic feedback. METHODS: The participants were divided into three groups: novices (0-50 laparoscopic procedures), intermediates (51-300 laparoscopic procedures), and experts (more than 300 procedures). It was assumed that psychomotor skills increase with experience. All participants conducted the tasks lifting and grasping and fine dissection 20 times each. Validity of the training setup was investigated by comparing the number of times each participant passed a predefined threshold level for a set of 19 parameters. RESULTS: Construct validity was established for one parameter; "misses on right side" on the lifting and grasping task, whereas the other 18 parameters did not show construct validity. CONCLUSION: The setup employed in this study failed to establish construct validity for more than one parameter. This indicates that the simulation of haptic feedback influences the training performance on laparoscopic simulators and is an important part of validating a training setup. A haptic device should generate haptic sensations in a realistic manner, without introducing frictional forces that are not inherent to laparoscopy.

Learning deep abdominal CT registration through adaptive loss weighting and synthetic data generation.

PURPOSE: This study aims to explore training strategies to improve convolutional neural network-based image-to-image deformable registration for abdominal imaging. METHODS: Different training strategies, loss functions, and transfer learning schemes were considered. Furthermore, an augmentation layer which generates artificial training image pairs on-the-fly was proposed, in addition to a loss layer that enables dynamic loss weighting. RESULTS: Guiding registration using segmentations in the training step proved beneficial for deep-learning-based image registration. Finetuning the pretrained model from the brain MRI dataset to the abdominal CT dataset further improved performance on the latter application, removing the need for a large dataset to yield satisfactory performance. Dynamic loss weighting also marginally improved performance, all without impacting inference runtime. CONCLUSION: Using simple concepts, we improved the performance of a commonly used deep image registration architecture, VoxelMorph. In future work, our framework, DDMR, should be validated on different datasets to further assess its value.

Prediction of guidewire-induced aortic deformations during EVAR: a finite element and in vitro study.

Introduction and aims: During an Endovascular Aneurysm Repair (EVAR) procedure a stiff guidewire is inserted from the iliac arteries. This induces significant deformations on the vasculature, thus, affecting the pre-operative planning, and the accuracy of image fusion. The aim of the present work is to predict the guidewire induced deformations using a finite element approach validated through experiments with patient-specific additive manufactured models. The numerical approach herein developed could improve the pre-operative planning and the intra-operative navigation. Material and methods: The physical models used for the experiments in the hybrid operating room, were manufactured from the segmentations of pre-operative Computed Tomography (CT) angiographies. The finite element analyses (FEA) were performed with LS-DYNA Explicit. The material properties used in finite element analyses were obtained by uniaxial tensile tests. The experimental deformed configurations of the aorta were compared to those obtained from FEA. Three models, obtained from Computed Tomography acquisitions, were investigated in the present work: A) without intraluminal thrombus (ILT), B) with ILT, C) with ILT and calcifications. Results and discussion: A good agreement was found between the experimental and the computational studies. The average error between the final in vitro vs. in silico aortic configurations, i.e., when the guidewire is fully inserted, are equal to 1.17, 1.22 and 1.40 mm, respectively, for Models A, B and C. The increasing trend in values of deformations from Model A to Model C was noticed both experimentally and numerically. The presented validated computational approach in combination with a tracking technology of the endovascular devices may be used to obtain the intra-operative configuration of the vessels and devices prior to the procedure, thus limiting the radiation exposure and the contrast agent dose.