Improving Valvular Pathologies and Ventricular Dysfunction Diagnostic Efficiency Using Combined Auscultation and Electrocardiography Data: A Multimodal AI Approach.

Simple sensor-based procedures, including auscultation and electrocardiography (ECG), can facilitate early diagnosis of valvular diseases, resulting in timely treatment. This study assessed the impact of combining these sensor-based procedures with machine learning on diagnosing valvular abnormalities and ventricular dysfunction. Data from auscultation at three distinct locations and 12-lead ECGs were collected from 1052 patients undergoing echocardiography. An independent cohort of 103 patients was used for clinical validation. These patients were screened for severe aortic stenosis (AS), severe mitral regurgitation (MR), and left ventricular dysfunction (LVD) with ejection fractions ≤ 40%. Optimal neural networks were identified by a fourfold cross-validation training process using heart sounds and various ECG leads, and their outputs were combined using a stacking technique. This composite sensor model had high diagnostic efficiency (area under the receiver operating characteristic curve (AUC) values: AS, 0.93; MR, 0.80; LVD, 0.75). Notably, the contribution of individual sensors to disease detection was found to be disease-specific, underscoring the synergistic potential of the sensor fusion approach. Thus, machine learning models that integrate auscultation and ECG can efficiently detect conditions typically diagnosed via imaging. Moreover, this study highlights the potential of multimodal artificial intelligence applications.

Efficient screening for severe aortic valve stenosis using understandable artificial intelligence: a prospective diagnostic accuracy study.

Aims: The medical need for screening of aortic valve stenosis (AS), which leads to timely and appropriate medical intervention, is rapidly increasing because of the high prevalence of AS in elderly population. This study aimed to establish a screening method using understandable artificial intelligence (AI) to detect severe AS based on heart sounds and to package the built AI into a smartphone application. Methods and results: In this diagnostic accuracy study, we developed multiple convolutional neural networks (CNNs) using a modified stratified five-fold cross-validation to detect severe AS in electronic heart sound data recorded at three auscultation locations. Clinical validation was performed with the developed smartphone application in an independent cohort (model establishment: n = 556, clinical validation: n = 132). Our ensemble technique integrating the heart sounds from multiple auscultation locations increased the detection accuracy of CNN model by compensating detection errors. The established smartphone application achieved a sensitivity, specificity, accuracy, and F1 value of 97.6% (41/42), 94.4% (85/90), 95.7% (126/132), and 0.93, respectively, which were higher compared with the consensus of cardiologists (81.0%, 93.3%, 89.4%, and 0.829, respectively), implying a good utility for severe AS screening. The Gradient-based Class Activation Map demonstrated that the built AIs could focus on specific heart sounds to differentiate the severity of AS. Conclusions: Our CNN model combining multiple auscultation locations and exported on smartphone application could efficiently identify severe AS based on heart sounds. The visual explanation of AI decisions for heart sounds was interpretable. These technologies may support medical training and remote consultations.