Fully Automated Echocardiogram Interpretation in Clinical Practice.

BACKGROUND: Automated cardiac image interpretation has the potential to transform clinical practice in multiple ways, including enabling serial assessment of cardiac function by nonexperts in primary care and rural settings. We hypothesized that advances in computer vision could enable building a fully automated, scalable analysis pipeline for echocardiogram interpretation, including (1) view identification, (2) image segmentation, (3) quantification of structure and function, and (4) disease detection. METHODS: Using 14 035 echocardiograms spanning a 10-year period, we trained and evaluated convolutional neural network models for multiple tasks, including automated identification of 23 viewpoints and segmentation of cardiac chambers across 5 common views. The segmentation output was used to quantify chamber volumes and left ventricular mass, determine ejection fraction, and facilitate automated determination of longitudinal strain through speckle tracking. Results were evaluated through comparison to manual segmentation and measurements from 8666 echocardiograms obtained during the routine clinical workflow. Finally, we developed models to detect 3 diseases: hypertrophic cardiomyopathy, cardiac amyloid, and pulmonary arterial hypertension. RESULTS: Convolutional neural networks accurately identified views (eg, 96% for parasternal long axis), including flagging partially obscured cardiac chambers, and enabled the segmentation of individual cardiac chambers. The resulting cardiac structure measurements agreed with study report values (eg, median absolute deviations of 15% to 17% of observed values for left ventricular mass, left ventricular diastolic volume, and left atrial volume). In terms of function, we computed automated ejection fraction and longitudinal strain measurements (within 2 cohorts), which agreed with commercial software-derived values (for ejection fraction, median absolute deviation=9.7% of observed, N=6407 studies; for strain, median absolute deviation=7.5%, n=419, and 9.0%, n=110) and demonstrated applicability to serial monitoring of patients with breast cancer for trastuzumab cardiotoxicity. Overall, we found automated measurements to be comparable or superior to manual measurements across 11 internal consistency metrics (eg, the correlation of left atrial and ventricular volumes). Finally, we trained convolutional neural networks to detect hypertrophic cardiomyopathy, cardiac amyloidosis, and pulmonary arterial hypertension with C statistics of 0.93, 0.87, and 0.85, respectively. CONCLUSIONS: Our pipeline lays the groundwork for using automated interpretation to support serial patient tracking and scalable analysis of millions of echocardiograms archived within healthcare systems.

The Feasibility and Usability of RunningCoach: A Remote Coaching System for Long-Distance Runners.

Studies have shown that about half of the injuries sustained during long-distance running involve the knee. Cadence (steps per minute) has been identified as a factor that is strongly associated with these running-related injuries, making it a worthwhile candidate for further study. As such, it is critical for long-distance runners to minimize their risk of injury by running at an appropriate running cadence. In this paper, we present the results of a study on the feasibility and usability of RunningCoach, a mobile health (mHealth) system that remotely monitors running cadence levels of runners in a continuous fashion, among other variables, and provides immediate feedback to runners in an effort to help them optimize their running cadence.

Upper extremity 3-dimensional reachable workspace assessment in amyotrophic lateral sclerosis by Kinect sensor.

INTRODUCTION: Reachable workspace is a measure that provides clinically meaningful information regarding arm function. In this study, a Kinect sensor was used to determine the spectrum of 3-dimensional reachable workspace encountered in a cross-sectional cohort of individuals with amyotrophic lateral sclerosis (ALS). METHODS: Bilateral 3D reachable workspace was recorded from 10 subjects with ALS and 17 healthy controls. The data were normalized by each individual's arm length to obtain a reachable workspace relative surface area (RSA). Concurrent validity was assessed by correlation with scoring on the ALS Functional Rating Score-revised (ALSFRSr). RESULTS: The Kinect-measured reachable workspace RSA differed significantly between the ALS and control subjects (0.579 ± 0.226 vs. 0.786 ± 0.069; P < 0.001). The RSA demonstrated correlation with ALSFRSr upper extremity items (Spearman correlation ρ = 0.569; P = 0.009). With worsening upper extremity function, as categorized by the ALSFRSr, the reachable workspace also decreased progressively. CONCLUSIONS: This study demonstrates the feasibility and potential of using a novel Kinect-based reachable workspace outcome measure in ALS.

Reachable workspace and performance of upper limb (PUL) in duchenne muscular dystrophy.

INTRODUCTION: The Kinect-based reachable workspace relative surface area (RSA) is compared with the performance of upper limb (PUL) assessment in Duchenne muscular dystrophy (DMD). METHODS: 29 individuals with DMD (ages: 7-23; Brooke: 1-5) underwent both Kinect-based reachable workspace RSA and PUL assessments. RSAs were also collected from 24 age-matched controls. Total and quadrant RSAs were compared with the PUL total, shoulder-, middle-, and distal-dimension scores. RESULTS: The total reachable workspace RSA correlated well with the total PUL score (Spearman ρ = -0.602; P < 0.001), and with each of the PUL dimensional scores: shoulder (ρ = -0.624; P < 0.001), middle (ρ = -0.564; P = 0.001), and distal (ρ = -0.630; P < 0.001). With quadrant RSA, reachability in a particular quadrant was closely associated with respective PUL dimensional-level function (lateral-upper quadrant for shoulder-, lateral-upper/lower quadrants for middle-, and lateral-lower quadrant for distal-level function). CONCLUSIONS: This study demonstrates concurrent validity of the reachable workspace outcome measure (RSA) with the DMD-specific upper extremity outcome measure (PUL).

Reachable workspace in facioscapulohumeral muscular dystrophy (FSHD) by Kinect.

INTRODUCTION: A depth-ranging sensor (Kinect) based upper extremity motion analysis system was applied to determine the spectrum of reachable workspace encountered in facioscapulohumeral muscular dystrophy (FSHD). METHODS: Reachable workspaces were obtained from 22 individuals with FSHD and 24 age- and height-matched healthy controls. To allow comparison, total and quadrant reachable workspace relative surface areas (RSAs) were obtained by normalizing the acquired reachable workspace by each individual's arm length. RESULTS: Significantly contracted reachable workspace and reduced RSAs were noted for the FSHD cohort compared with controls (0.473 ± 0.188 vs. 0.747 ± 0.082; P < 0.0001). With worsening upper extremity function as categorized by the FSHD evaluation subscale II + III, the upper quadrant RSAs decreased progressively, while the lower quadrant RSAs were relatively preserved. There were no side-to-side differences in reachable workspace based on hand-dominance. CONCLUSIONS: This study demonstrates the feasibility and potential of using an innovative Kinect-based reachable workspace outcome measure in FSHD.

Upper extremity 3-dimensional reachable workspace analysis in dystrophinopathy using Kinect.

INTRODUCTION: An innovative upper extremity 3-dimensional (3D) reachable workspace outcome measure acquired using the Kinect sensor is applied toward Duchenne/Becker muscular dystrophy (DMD/BMD). The validity, sensitivity, and clinical meaningfulness of this novel outcome measure are examined. METHODS: Upper extremity function assessment (Brooke scale and NeuroQOL questionnaire) and Kinect-based reachable workspace analyses were conducted in 43 individuals with dystrophinopathy (30 DMD and 13 BMD, aged 7-60 years) and 46 controls (aged 6-68 years). RESULTS: The reachable workspace measure reliably captured a wide range of upper extremity impairments encountered in both pediatric and adult, as well as ambulatory and non-ambulatory individuals with dystrophinopathy. Reduced reachable workspaces were noted for the dystrophinopathy cohort compared with controls, and they correlated with Brooke grades. In addition, progressive reduction in reachable workspace correlated directly with worsening ability to perform activities of daily living, as self-reported on the NeuroQOL. CONCLUSION: This study demonstrates the utility and potential of the novel sensor-acquired reachable workspace outcome measure in dystrophinopathy.

Reachable workspace reflects dynamometer-measured upper extremity strength in facioscapulohumeral muscular dystrophy.

INTRODUCTION: It is not known whether a reduction in reachable workspace closely reflects loss of upper extremity strength in facioscapulohumeral muscular dystrophy (FSHD). In this study we aimed to determine the relationship between reachable workspace and quantitative upper extremity strength measures. METHODS: Maximal voluntary isometric contraction (MVIC) testing of bilateral elbow flexion and shoulder abduction by hand-held dynamometry was performed on 26 FSHD and 27 control subjects. In addition, Kinect sensor-based 3D reachable workspace relative surface areas (RSAs) were obtained. Loading (500-g weight) effects on reachable workspace were also evaluated. RESULTS: Quantitative upper extremity strength (MVIC of elbow flexion and shoulder abduction) correlated with Kinect-acquired reachable workspace RSA (R = 0.477 for FSHD, P = 0.0003; R = 0.675 for the combined study cohort, P < 0.0001). Progressive reduction in RSA reflected worsening MVIC measures. Loading impacted the moderately weak individuals the most with additional reductions in RSA. CONCLUSIONS: Reachable workspace outcome measure is reflective of upper extremity strength impairment in FSHD.

Recognizing the intensity of strength training exercises with wearable sensors.

In this paper we propose a system based on a network of wearable accelerometers and an off-the-shelf smartphone to recognize the intensity of stationary activities, such as strength training exercises. The system uses a hierarchical algorithm, consisting of two layers of Support Vector Machines (SVMs), to first recognize the type of exercise being performed, followed by recognition of exercise intensity. The first layer uses a single SVM to recognize the type of the performed exercise. Based on the recognized type a corresponding intensity prediction SVM is selected on the second layer, specializing in intensity prediction for the recognized type of exercise. We evaluate the system for a set of upper-body exercises using different weight loads. Additionally, we compare the most important features for exercise and intensity recognition tasks and investigate how different sliding window combinations, sensor configurations and number of training subjects impact the algorithm performance. We perform all of the experiments for two different types of features to evaluate the feasibility of implementation on resource constrained hardware. The results show the algorithm is able to recognize exercise types with approximately 85% accuracy and 6% intensity prediction error. Furthermore, due to similar performance using different types of features, the algorithm offers potential for implementation on resource constrained hardware.

Architecture of an automated coaching system for elderly population.

We present an automated coaching system for elderly population living in assisted homes. The system guides its users through a sequence of exercises and tests. Each exercise is demonstrated by a pre-recorded video of a coach, checked for correct execution and qualitatively evaluated. Automatic coaching advices are generated in order to improve the execution. Performance measurements are shown as an immediate feedback to the user, and stored and evaluated over time. The system is designed to allow for a remote interaction with a coach, and, to bolster social aspect of the exercise, for concurrent exercise of two (or eventually multiple) remote users.

Upper extremity reachable workspace evaluation with Kinect.

We propose a novel low-cost method for quantitative assessment of upper extremity workspace envelope using Microsoft Kinect camera. In clinical environment there are currently no practical and cost-effective methods available to provide arm-function evaluation in three-dimensional space. In this paper we examine the accuracy of the proposed technique for workspace estimation using Kinect in comparison with a motion capture system. The experimental results show that the developed system is capable of capturing the workspace with sufficient accuracy and robustness.

mHealth application for upper extremity range of motion and reachable workspace.

We present mobile health (mHealth) applications utilizing embedded phone sensors as an angle-measuring device for upper-limb range of motion (ROM) and estimation of reachable workspace to assist in evaluation of upper limb functional capacity. Our results show that the phone can record accurate measurements, as well as provide additional functionalities for clinicians.

Real-time human pose detection and tracking for tele-rehabilitation in virtual reality.

We present a real-time algorithm for human pose detection and tracking from vision-based 3D data and its application to tele-rehabilitation in virtual environments. We employ stereo camera(s) to capture 3D avatars of geographically dislocated patient and therapist in real-time, while sending the data remotely and displaying it in a virtual scene. A pose detection and tracking algorithm extracts kinematic parameters from each participant and determines pose similarity. The pose similarity score is used to quantify patient's performance and provide real-time feedback for remote rehabilitation.

Real-time 3D avatars for tele-rehabilitation in virtual reality.

We present work in progress on a tele-immersion system for telerehabilitation using real-time stereo vision and virtual environments. Stereo reconstruction is used to capture user's 3D avatar in real time and project it into a shared virtual environment, enabling a patient and therapist to interact remotely. Captured data can also be used to analyze the movement and provide feedback to the patient as we present in a preliminary study of stepping-in-place task. Such tele-presence system could in the future allow patients to interact remotely with remote physical therapist and virtual environment while objectively tracking their performance.

Toward Real-Time Muscle Force Inference and Device Control via Optical-Flow-Tracked Muscle Deformation.

Despite the utility of musculoskeletal dynamics modeling, there exists no safe, noninvasive method of measuring in vivo muscle output force in real time - limiting both biomechanical insight into dexterous motion and intuitive control of assistive devices. In this paper, we demonstrate that muscle deformation constitutes a promising, yet unexplored signal from which to 1) infer such forces and 2) build novel device control schemes. Through a case study of the elbow joint on a preliminary cohort of 10 subjects, we show that muscle deformation (specifically, thickness change of the brachioradialis, as measured via ultrasound and tracked via optical flow) correlates well with elbow output force to an extent comparable with standard surface electromyography (sEMG) activation during varied isometric elbow contraction. We then show that, given real-time visual feedback, subjects can readily perform a trajectory tracking task using this deformation signal, and that they largely prefer this method to a comparable sEMG-based control scheme and perform the tracking task with similar accuracy. Together, these contributions illustrate muscle deformation's potential utility for both biomechanical study of individual muscle dynamics and device control, in a manner that - thanks to, unlike sEMG, the localized nature of the signal and its tight mechanistic coupling to output force - is readily extensible to multiple muscles and device degrees of freedom. To enable such future extensions, all modeling, tracking, and visualization software described in this paper, as well as all raw and processed data, have been made available on SimTK as part of the Open-Arm project (https://simtk.org/projects/openarm) for general research use.

2K09 and thereafter : the coming era of integrative bioinformatics, systems biology and intelligent computing for functional genomics and personalized medicine research.

Significant interest exists in establishing synergistic research in bioinformatics, systems biology and intelligent computing. Supported by the United States National Science Foundation (NSF), International Society of Intelligent Biological Medicine (http://www.ISIBM.org), International Journal of Computational Biology and Drug Design (IJCBDD) and International Journal of Functional Informatics and Personalized Medicine, the ISIBM International Joint Conferences on Bioinformatics, Systems Biology and Intelligent Computing (ISIBM IJCBS 2009) attracted more than 300 papers and 400 researchers and medical doctors world-wide. It was the only inter/multidisciplinary conference aimed to promote synergistic research and education in bioinformatics, systems biology and intelligent computing. The conference committee was very grateful for the valuable advice and suggestions from honorary chairs, steering committee members and scientific leaders including Dr. Michael S. Waterman (USC, Member of United States National Academy of Sciences), Dr. Chih-Ming Ho (UCLA, Member of United States National Academy of Engineering and Academician of Academia Sinica), Dr. Wing H. Wong (Stanford, Member of United States National Academy of Sciences), Dr. Ruzena Bajcsy (UC Berkeley, Member of United States National Academy of Engineering and Member of United States Institute of Medicine of the National Academies), Dr. Mary Qu Yang (United States National Institutes of Health and Oak Ridge, DOE), Dr. Andrzej Niemierko (Harvard), Dr. A. Keith Dunker (Indiana), Dr. Brian D. Athey (Michigan), Dr. Weida Tong (FDA, United States Department of Health and Human Services), Dr. Cathy H. Wu (Georgetown), Dr. Dong Xu (Missouri), Drs. Arif Ghafoor and Okan K Ersoy (Purdue), Dr. Mark Borodovsky (Georgia Tech, President of ISIBM), Dr. Hamid R. Arabnia (UGA, Vice-President of ISIBM), and other scientific leaders. The committee presented the 2009 ISIBM Outstanding Achievement Awards to Dr. Joydeep Ghosh (UT Austin), Dr. Aidong Zhang (Buffalo) and Dr. Zhi-Hua Zhou (Nanjing) for their significant contributions to the field of intelligent biological medicine.

Design of a Passive, Variable Stiffness Exoskeleton for Triceps Deficiency Mitigation.

Individuals with neurological impairment, particularly those with cervical level spinal cord injuries (SCI), often have difficulty with daily tasks due to triceps weakness or total loss of function. More demanding tasks, such as sit-skiing, may be rendered impossible due to their extreme strength demands. Design of exoskeletons that address this issue by providing supplemental strength in arm extension is an active field of research but commercial devices are not yet available for use. Most current designs employ electric motors that necessitate the addition of bulky power sources and extraneous wiring, rendering the devices impractical in daily life. The possibility of powering an upper extremity exoskeleton passively has been explored, but to date, none have delivered sufficient function or strength to provide useful assistance for sit-skiing. We seek to rectify this with the design of a passively actuated exoskeletal arm brace capable of operating in two, adjustable-strength modes: one for low level gravity compensation to aid in active range of motion, and the other for more stringent weight bearing activities. The mechanism developed through this paper allows for an affordable, lightweight, modular device that can be adjusted and customized for the needs of each individual patient.

Reachable Workspace and Proximal Function Measures for Quantifying Upper Limb Motion.

There are a lack of quantitative measures for clinically assessing upper limb function. Conventional biomechanical performance measures are restricted to specialist labs due to hardware cost and complexity, while the resulting measurements require specialists for analysis. Depth cameras are low cost and portable systems that can track surrogate joint positions. However, these motions may not be biologically consistent, which can result in noisy, inaccurate movements. This paper introduces a rigid body modelling method to enforce biological feasibility of the recovered motions. This method is evaluated on an existing depth camera assessment: the reachable workspace (RW) measure for assessing gross shoulder function. As a rigid body model is used, position estimates of new proximal targets can be added, resulting in a proximal function (PF) measure for assessing a subject's ability to touch specific body landmarks. The accuracy, and repeatability of these measures is assessed on ten asymptomatic subjects, with and without rigid body constraints. This analysis is performed both on a low-cost depth camera system and a gold-standard active motion capture system. The addition of rigid body constraints was found to improve accuracy and concordance of the depth camera system, particularly in lateral reaching movements. Both RW and PF measures were found to be feasible candidates for clinical assessment, with future analysis needed to determine their ability to detect changes within specific patient populations.

Teleimmersive environment for remote medical collaboration.

We present the work in progress on teleimmersive framework that would allow doctors to collaborate in more natural way by being immersed in the virtual space along with the medical data they could examine and discuss with remotely located colleagues. The system consists of multiple cameras, hand tracking with gesture recognition, and virtual reality system to support volume visualization and 3D interaction with medical data.

Kinematic and Kinetic Validation of an Improved Depth Camera Motion Assessment System Using Rigid Bodies.

The study of joint kinematics and dynamics has broad clinical applications, including the identification of pathological motions or compensation strategies and the analysis of dynamic stability. High-end motion capture systems, however, are expensive and require dedicated camera spaces with lengthy setup and data processing commitments. Depth cameras, such as the Microsoft Kinect, provide an inexpensive, marker-free alternative at the sacrifice of joint-position accuracy. In this work, we present a fast framework for adding biomechanical constraints to the joint estimates provided by a depth camera system. We also present a new model for the lower lumbar joint angle. We validate key joint position, angle, and velocity measurements against a gold standard active motion-capture system on ten healthy subjects performing sit to stand (STS). Our method showed significant improvement in mean absolute error and intraclass correlation coefficients for the recovered joint angles and position-based metrics. These improvements suggest that depth cameras can provide an accurate and clinically viable method of rapidly assessing the kinematics and kinetics of the STS action, providing data for further analysis using biomechanical or machine learning methods.

Estimating Sit-to-Stand Dynamics Using a Single Depth Camera.

Kinetic and dynamic motion analysis provides quantitative, functional assessments of human ability that are unobtainable through static imaging methods or subjective surveys. While biomechanics facilities are equipped to perform this measurement and analysis, the clinical translation of these methods is limited by the specialized skills and equipment needed. This paper presents and validates a method for estimating dynamic effects such as joint torques and body momenta using a single depth camera. An allometrically scaled, sagittal plane dynamic model is used to estimate the joint torques at the ankles, knees, hips, and low back, as well as the torso momenta, and shear and normal loads at the L5-S1 disk. These dynamic metrics are applied to the sit-to-stand motion and validated against a gold-standard biomechanical system consisting of full-body active motion-capture and force-sensing systems. The metrics obtained from the proposed method were found to have excellent concordance with peak metrics that are consistent with prior biomechanical studies. This suggests the feasibility of using this system for rapid clinical assessment, with applications in diagnostics, longitudinal tracking, and quantifying patient recovery.