Safe contact-based robot active search using Bayesian optimization and control barrier functions.

In robotics, active exploration and learning in uncertain environments must take into account safety, as the robot may otherwise damage itself or its surroundings. This paper presents a method for safe active search using Bayesian optimization and control barrier functions. As robot paths undertaken during sampling are continuous, we consider an informative continuous expected improvement acquisition function. To safely bound the contact forces between the robot and its surroundings, we leverage exponential control barrier functions, utilizing the derivative of the force in the contact model to increase robustness to uncertainty in the contact boundary. Our approach is demonstrated on a fully autonomous robot for ultrasound scanning of rheumatoid arthritis (RA). Here, active search is a critical component of ensuring high image quality. Furthermore, bounded contact forces between the ultrasound probe and the patient ensure patient safety and better scan quality. To the best of our knowledge, our results are both the first demonstration of safe active search on a fully autonomous robot for ultrasound scanning of rheumatoid arthritis and the first experimental evaluation of bounding contact forces in the context of medical robotics using control barrier functions. The results show that when search time is limited to less than 60 s, informative continuous expected improvement leads to a 92% success, a 13% improvement compared to expected improvement. Meanwhile, exponential control barrier functions can limit the force applied by the robot to under 5 N, even in cases where the contact boundary is specified incorrectly by -1 or +4 mm.

Artificial intelligence model for segmentation and severity scoring of osteophytes in hand osteoarthritis on ultrasound images.

Objective: To develop an artificial intelligence (AI) model able to perform both segmentation of hand joint ultrasound images for osteophytes, bone, and synovium and perform osteophyte severity scoring following the EULAR-OMERACT grading system (EOGS) for hand osteoarthritis (OA). Methods: One hundred sixty patients with pain or reduced function of the hands were included. Ultrasound images of the metacarpophalangeal (MCP), proximal interphalangeal (PIP), distal interphalangeal (DIP), and first carpometacarpal (CMC1) joints were then manually segmented for bone, synovium and osteophytes and scored from 0 to 3 according to the EOGS for OA. Data was divided into a training, validation, and test set. The AI model was trained on the training data to perform bone, synovium, and osteophyte identification on the images. Based on the manually performed image segmentation, an AI was trained to classify the severity of osteophytes according to EOGS from 0 to 3. Percent Exact Agreement (PEA) and Percent Close Agreement (PCA) were assessed on individual joints and overall. PCA allows a difference of one EOGS grade between doctor assessment and AI. Results: A total of 4615 ultrasound images were used for AI development and testing. The developed AI model scored on the test set for the MCP joints a PEA of 76% and PCA of 97%; for PIP, a PEA of 70% and PCA of 97%; for DIP, a PEA of 59% and PCA of 94%, and CMC a PEA of 50% and PCA of 82%. Combining all joints, we found a PEA between AI and doctor assessments of 68% and a PCA of 95%. Conclusion: The developed AI model can perform joint ultrasound image segmentation and severity scoring of osteophytes, according to the EOGS. As proof of concept, this first version of the AI model is successful, as the agreement performance is slightly higher than previously found agreements between experts when assessing osteophytes on hand OA ultrasound images. The segmentation of the image makes the AI explainable to the doctor, who can immediately see why the AI applies a given score. Future validation in hand OA cohorts is necessary though.

Retinal main vessel calibers and systemic markers for long-term development of proliferative diabetic retinopathy.

PURPOSE: To evaluate if retinal vascular calibers and systemic risk factors in patients with no or minimal diabetic retinopathy (DR) can predict risk of long-term progression to proliferative diabetic retinopathy (PDR). METHODS: This was a matched case-control study of patients with diabetes having no or minimal DR at baseline with (cases) or without (controls) subsequent development of PDR. We collected six-field, 45-degree retinal images, demographic and clinical data from the Funen Diabetes Database. RESULTS: We included 52 eyes from 39 cases and 107 eyes from 89 controls matched on sex, age, type of diabetes, time from first to last screening episode and baseline DR level. Cases had higher HbA1c (73 vs. 55 mmoL/moL; p < 0.001), triglycerides (1.32 vs. 1.16 mmoL/L; p = 0.02) and longer duration of diabetes (19 vs. 14 years; p = 0.01), but the groups did not differ in calibers of retinal arterioles (229 vs. 227 µm; p = 0.49), venules (289 vs. 290 µm; p = 0.83) or the arterio-to-venule ratio (0.78 vs. 0.77; p = 0.86).In a multivariable logistic regression model with cluster robust standard error, HbA1c (OR 1.54 per 10 mmoL/moL; 95%-CI: 1.15-2.07; p = 0.004), triglyceride (OR 1.39 per 1 mmoL/L; 95%-CI: 1.03-1.86; p = 0.03) and duration of diabetes (OR 1.09 per year; 95%-CI: 1.03-1.16; p = 0.003) were independent risk factors for PDR. CONCLUSION: Retinal vascular calibers did not predict long-term development of PDR in contrast to well-established risk factors like HbA1c, triglyceride and duration of diabetes.

Five-Year Incidence of Proliferative Diabetic Retinopathy and Associated Risk Factors in a Nationwide Cohort of 201 945 Danish Patients with Diabetes.

Purpose: To evaluate the proliferative diabetic retinopathy (PDR) progression rates and identify the demographic and clinical characteristics of patients who later developed PDR compared with patients who did not progress to that state. Design: A national 5-year register-based cohort study including 201 945 patients with diabetes. Subjects: Patients with diabetes who had attended the Danish national screening program (2013-2018) for diabetic retinopathy (DR). Methods: We used the first screening episode as the index date and included both eyes of patients with and without subsequent progression of PDR. Data were linked with various national health registries to investigate relevant clinical and demographic parameters. The International Clinical Retinopathy Disease Scale was used to classify DR, with no DR as level 0, mild DR as level 1, moderate DR as level 2, severe DR as level 3, and PDR as level 4. Main Outcome Measures: Hazard ratios (HRs) for incident PDR for all relevant demographic and clinical parameters and 1-, 3-, and 5-year incidence rates of PDR according to baseline DR level. Results: Progression to PDR within 5 years was identified in 2384 eyes of 1780 patients. Proliferative diabetic retinopathy progression rates from baseline DR level 3 at 1, 3 and 5 years were 3.6%, 10.9%, and 14.7%, respectively. The median number of visits was 3 (interquartile range, 1-4). Progression to PDR was predicted in a multivariable model by duration of diabetes (HR, 4.66 per 10 years; 95% confidence interval [CI], 4.05-5.37), type 1 diabetes (HR, 9.61; 95% CI, 8.01-11.53), a Charlson Comorbidity Index score of > 0 (score 1: HR, 4.62; 95% CI, 4.14-5.15; score 2: HR, 2.28; 95% CI, 1.90-2.74; score ≥ 3: HR, 4.28; 95% CI, 3.54-5.17), use of insulin (HR, 5.33; 95% CI, 4.49-6.33), and use of antihypertensive medications (HR, 2.23; 95% CI, 1.90-2.61). Conclusions: In a 5-year longitudinal study of an entire screening nation, we found increased risk of PDR with increasing baseline DR levels, longer duration of diabetes, type 1 diabetes, systemic comorbidity, use of insulin, and blood pressure-lowering medications. Most interestingly, we found lower risk of progression from DR level 3 to PDR compared with that in previous studies. Financial Disclosures: Proprietary or commercial disclosure may be found after the references.

Ultrasound joint examination by an automated system versus by a rheumatologist: from a patient perspective.

BACKGROUND: The Arthritis Ultrasound Robot (ARTHUR) is an automated system for ultrasound scanning of the joints of both hands and wrists, with subsequent disease activity scoring using artificial intelligence. The objective was to describe the patient's perspective of being examined by ARTHUR, compared to an ultrasound examination by a rheumatologist. Further, to register any safety issues with the use of ARTHUR. METHODS: Twenty-five patients with rheumatoid arthritis (RA) had both hands and wrists examined by ultrasound, first by a rheumatologist and subsequently by ARTHUR. Patient-reported outcomes (PROs) were obtained after the examination by the rheumatologist and by ARTHUR. PROs regarding pain, discomfort and overall experience were collected, including willingness to be examined again by ARTHUR as part of future clinical follow-up. All ARTHUR examinations were observed for safety issues. RESULTS: There was no difference in pain or discomfort between the examination by a rheumatologist and by ARTHUR (p = 0.29 and p = 0.20, respectively). The overall experience of ARTHUR was described as very good or good by 92% (n = 23), with no difference compared to the examination by the rheumatologist (p = 0.50). All (n = 25) patients were willing to be examined by ARTHUR again, and 92% (n = 23) would accept ARTHUR as a regular part of their RA clinical follow up. No safety issues were registered. CONCLUSIONS: Joint ultrasound examination by ARTHUR was safe and well-received, with no difference in PRO components compared to ultrasound examination by a rheumatologist. Fully automated systems for RA disease activity assessment could be important in future strategies for managing RA patients. TRIAL REGISTRATION: The study was evaluated by the regional ethics committee (ID: S-20200145), which ruled it was not a clinical trial necessary for their approval. It was a quality assessment project, as there was no intervention to the patient. The study was hereafter submitted and registered to Odense University Hospital, Region of Southern Denmark as a quality assessment project and approved (ID: 20/55294).

Monopolar, bipolar, tripolar, and tetrapolar configurations in robot assisted electrical impedance sensing.

Objective.Tissue recognition is a critical process during a Robot-assisted minimally invasive surgery (RMIS) and it relies on the involvement of advanced sensing technology.Approach.In this paper, the concept of Robot Assisted Electrical Impedance Sensing (RAEIS) is utilized and further developed aiming to sense the electrical bioimpedance of target tissue directly based on the existing robotic instruments and control strategy. Specifically, we present a new sensing configuration called pseudo-tetrapolar method. With the help of robotic control, we can achieve a similar configuration as traditional tetrapolar, and with better accuracy.Main results.Five configurations including monopolar, bipolar, tripolar, tetrapolar and pseudo-tetrapolar are analyzed and compared through simulation experiments. Advantages and disadvantages of each configuration are thus discussed.Significance.This study investigates the measurement of tissue electrical property directly based on the existing robotic surgical instruments. Specifically, different sensing configurations can be realized through different connection and control strategies, making them suitable for different application scenarios.

A digital online platform for education and certification of diabetic retinopathy health care professionals in the Region of Southern Denmark.

PURPOSE: The incidence of diabetes continues to increase across the world. As the number of patients rises, so does the need for educated health care professionals. Diabetic retinopathy (DR) remains one of the primary complications in diabetes, and screening has proved to be a cost-effective measure to avoid DR-related blindness. Denmark has an established screening programme, but no formal training of the people responsible for analysing retinal images. METHODS: We here present an online learning platform that offers a diabetic eye screening course for health care professionals undertaking screening responsibility in the Region of Southern Denmark. The course is divided into lectures, each focussed on identifying different levels of DR or detecting related lesions. The course is free to use on-demand, contains instructional videos, interactive tests and exercises, and it is concluded with a certification test. The tools on the platform can in addition be used to generate data for research purposes, such as comparing users or experts in detection of lesions or annotating data for the development of machine learning models. RESULTS: More than 150 participants have so far completed the course, and the platform is being adopted for education in other regions of Denmark.

An Electrical Bioimpedance Scanning System for Subsurface Tissue Detection in Robot Assisted Minimally Invasive Surgery.

In Robot Assisted Minimally Invasive Surgery, discriminating critical subsurface structures is essential to make the surgical procedure safer and more efficient. In this paper, a novel robot assisted electrical bio-impedance scanning (RAEIS) system is developed and validated using a series of experiments. The proposed system constructs a tri-polar sensing configuration for tissue homogeneity inspection. Specifically, two robotic forceps are used as electrodes for applying electric current and measuring reciprocal voltages relative to a ground electrode which is placed distal from the measuring site. Compared to existing electrical bioimpedance sensing technology, the proposed system is able to use miniaturized electrodes to measure a site flexibly with enhanced subsurfacial detection capability. This paper presents the concept, the modeling of the sensing method, the hardware design, and the system calibration. Subsequently, a series of experiments are conducted for system evaluation including finite element simulation, saline solution bath experiments and experiments based on ex vivo animal tissues. The experimental results demonstrate that the proposed system can measure the resistivity of the material with high accuracy, and detect a subsurface non-homogeneous object with 100% success rate. The proposed parameters estimation algorithm is able to approximate the resistivity and the depth of the subsurface object effectively with one fast scanning.

Ultrasound joint examination by an automated system versus by a rheumatologist: from a patient perspective

Abstract Background: The Arthritis Ultrasound Robot (ARTHUR) is an automated system for ultrasound scanning of the joints of both hands and wrists, with subsequent disease activity scoring using artificial intelligence. The objective was to describe the patient's perspective of being examined by ARTHUR, compared to an ultrasound examination by a rheumatologist. Further, to register any safety issues with the use of ARTHUR. Methods: Twenty-five patients with rheumatoid arthritis (RA) had both hands and wrists examined by ultrasound, first by a rheumatologist and subsequently by ARTHUR. Patient-reported outcomes (PROs) were obtained after the examination by the rheumatologist and by ARTHUR. PROs regarding pain, discomfort and overall experience were collected, including willingness to be examined again by ARTHUR as part of future clinical follow-up. All ARTHUR examinations were observed for safety issues. Results: There was no difference in pain or discomfort between the examination by a rheumatologist and by ARTHUR ( p =0.29 and p =0.20, respectively). The overall experience of ARTHUR was described as very good or good by 92% (n =23), with no difference compared to the examination by the rheumatologist ( p =0.50). All (n =25) patients were willing to be examined by ARTHUR again, and 92% (n =23) would accept ARTHUR as a regular part of their RA clinical follow up. No safety issues were registered. Conclusion: Joint ultrasound examination by ARTHUR was safe and well-received, with no difference in PRO components compared to ultrasound examination by a rheumatologist. Fully automated systems for RA disease activity assessment could be important in future strategies for managing RA patients. Trial registration: The study was evaluated by the regional ethics committee (ID: S-20200145), which ruled it was not a clinical trial necessary for their approval. It was a quality assessment project, as there was no intervention to the patient. The study was hereafter submitted and registered to Odense University Hospital, Region of Southern Denmark as a quality assessment project and approved (ID: 20/55294).

Robot-Assisted Electrical Impedance Scanning system for 2D Electrical Impedance Tomography tissue inspection.

The electrical impedance tomography (EIT) technology is an important medical imaging approach to show the electrical characteristics and the homogeneity of a tissue region noninvasively. Recently, this technology has been introduced to the Robot Assisted Minimally Invasive Surgery (RAMIS) for assisting the detection of surgical margin with relevant clinical benefits. Nevertheless, most EIT technologies are based on a fixed multiple-electrodes probe which limits the sensing flexibility and capability significantly. In this study, we present a method for acquiring the EIT measurements during a RAMIS procedure using two already existing robotic forceps as electrodes. The robot controls the forceps tips to a series of predefined positions for injecting excitation current and measuring electric potentials. Given the relative positions of electrodes and the measured electric potentials, the spatial distribution of electrical conductivity in a section view can be reconstructed. Realistic experiments are designed and conducted to simulate two tasks: subsurface abnormal tissue detection and surgical margin localization. According to the reconstructed images, the system is demonstrated to display the location of the abnormal tissue and the contrast of the tissues' conductivity with an accuracy suitable for clinical applications.

A disposable force regulation mechanism for throat swab robot.

Robots can protect healthcare workers from being infected by the COVID-19 and play a role in throat swab sampling operation. A critical requirement in this process is to maintain a constant force on the tissue for ensuring a safe and good sampling. In this study, we present the design of a disposable mechanism with two non-linear springs to achieve a 0.6 N constant force within a 20 mm displacement. The nonlinear spring is designed through optimization based on Finite Element Simulation and Genetic Algorithm. Prototype of the mechanism is made and tested. The experimental results show that the mechanism can provide 0.67±0.04 N and 0.57±0.02 N during its compression and return process. The proposed design can be extended to different scales and used in a variety of scenario where safe interacting with human is required.

Applying cascaded convolutional neural network design further enhances automatic scoring of arthritis disease activity on ultrasound images from rheumatoid arthritis patients.

OBJECTIVES: We have previously shown that neural network technology can be used for scoring arthritis disease activity in ultrasound images from rheumatoid arthritis (RA) patients, giving scores according to the EULAR-OMERACT grading system. We have now further developed the architecture of this neural network and can here present a new idea applying cascaded convolutional neural network (CNN) design with even better results. We evaluate the generalisability of this method on unseen data, comparing the CNN with an expert rheumatologist. METHODS: The images were graded by an expert rheumatologist according to the EULAR-OMERACT synovitis scoring system. CNNs were systematically trained to find the best configuration. The algorithms were evaluated on a separate test data set and compared with the gradings of an expert rheumatologist on a per-joint basis using a Kappa statistic, and on a per-patient basis using a Wilcoxon signed-rank test. RESULTS: With 1678 images available for training and 322 images for testing the model, it achieved an overall four-class accuracy of 83.9%. On a per-patient level, there was no significant difference between the classifications of the model and of a human expert (p=0.85). Our original CNN had a four-class accuracy of 75.0%. CONCLUSIONS: Using a new network architecture we have further enhanced the algorithm and have shown strong agreement with an expert rheumatologist on a per-joint basis and on a per-patient basis. This emphasises the potential of using CNNs with this architecture as a strong assistive tool for the objective assessment of disease activity of RA patients.

Neural networks for automatic scoring of arthritis disease activity on ultrasound images.

Background: The development of standardised methods for ultrasound (US) scanning and evaluation of synovitis activity by the OMERACT-EULAR Synovitis Scoring (OESS) system is a major step forward in the use of US in the diagnosis and monitoring of patients with inflammatory arthritis. The variation in interpretation of disease activity on US images can affect diagnosis, treatment and outcomes in clinical trials. We, therefore, set out to investigate if we could utilise neural network architecture for the interpretation of disease activity on Doppler US images, using the OESS scoring system. Methods: Two state-of-the-art neural networks were used to extract information from 1342 Doppler US images from patients with rheumatoid arthritis (RA). One neural network divided images as either healthy (Doppler OESS score 0 or 1) or diseased (Doppler OESS score 2 or 3). The other to score images across all four of the OESS systems Doppler US scores (0-3). The neural networks were hereafter tested on a new set of RA Doppler US images (n=176). Agreement between rheumatologist's scores and network scores was measured with the kappa statistic. Results: For the neural network assessing healthy/diseased score, the highest accuracies compared with an expert rheumatologist were 86.4% and 86.9% with a sensitivity of 0.864 and 0.875 and specificity of 0.864 and 0.864, respectively. The other neural network developed to four class Doppler OESS scoring achieved an average per class accuracy of 75.0% and a quadratically weighted kappa score of 0.84. Conclusion: This study is the first to show that neural network technology can be used in the scoring of disease activity on Doppler US images according to the OESS system.