Prospective Human Validation of Artificial Intelligence Interventions in Cardiology: A Scoping Review.

Background: Despite the potential of artificial intelligence (AI) in enhancing cardiovascular care, its integration into clinical practice is limited by a lack of evidence on its effectiveness with respect to human experts or gold standard practices in real-world settings. Objectives: The purpose of this study was to identify AI interventions in cardiology that have been prospectively validated against human expert benchmarks or gold standard practices, assessing their effectiveness, and identifying future research areas. Methods: We systematically reviewed Scopus and MEDLINE to identify peer-reviewed publications that involved prospective human validation of AI-based interventions in cardiology from January 2015 to December 2023. Results: Of 2,351 initial records, 64 studies were included. Among these studies, 59 (92.2%) were published after 2020. A total of 11 (17.2%) randomized controlled trials were published. AI interventions in 44 articles (68.75%) reported definite clinical or operational improvements over human experts. These interventions were mostly used in imaging (n = 14, 21.9%), ejection fraction (n = 10, 15.6%), arrhythmia (n = 9, 14.1%), and coronary artery disease (n = 12, 18.8%) application areas. Convolutional neural networks were the most common predictive model (n = 44, 69%), and images were the most used data type (n = 38, 54.3%). Only 22 (34.4%) studies made their models or data accessible. Conclusions: This review identifies the potential of AI in cardiology, with models often performing equally well as human counterparts for specific and clearly scoped tasks suitable for such models. Nonetheless, the limited number of randomized controlled trials emphasizes the need for continued validation, especially in real-world settings that closely examine joint human AI decision-making.

Comparing Clustering Methods Applied to Tinnitus within a Bootstrapped and Diagnostic-Driven Semi-Supervised Framework.

The understanding of tinnitus has always been elusive and is largely prevented by its intrinsic heterogeneity. To address this issue, scientific research has aimed at defining stable and easily identifiable subphenotypes of tinnitus. This would allow better disentangling the multiple underlying pathophysiological mechanisms of tinnitus. In this study, three-dimensionality reduction techniques and two clustering methods were benchmarked on a database of 2772 tinnitus patients in order to obtain a reliable segmentation of subphenotypes. In this database, tinnitus patients' endotypes (i.e., parts of a population with a condition with distinct underlying mechanisms) are reported when diagnosed by an ENT expert in tinnitus management. This partial labeling of the dataset enabled the design of an original semi-supervised framework. The objective was to perform a benchmark of different clustering methods to get as close as possible to the initial ENT expert endotypes. To do so, two metrics were used: a primary one, the quality of the separation of the endotypes already identified in the database, as well as a secondary one, the stability of the obtained clusterings. The relevance of the results was finally reviewed by two ENT experts in tinnitus management. A 20-cluster clustering was selected as the best-performing, the most-clinically relevant, and the most-stable through bootstrapping. This clustering used a T-SNE method as the dimensionality reduction technique and a k-means algorithm as the clustering method. The characteristics of this clustering are presented in this article.

Development and validation of an educational software based in artificial neural networks for training in radiology (JORCAD) through an interactive learning activity.

The use of Computer Aided Detection (CAD) software has been previously documented as a valuable tool to improve specialist training in Radiology. This research assesses the utility of an educational software tool aimed to train residents in Radiology and other related medical specialties and students from Medicine degree. This in-house developed software, called JORCAD, integrates a CAD system based in Convolutional Neural Networks (CNNs) with annotated cases from radiological image databases. The methodology followed for software validation was expert judgement after completing an interactive learning activity. Participants received a theoretical session and a software usage tutorial and afterwards utilized the application in a dedicated workstation to analyze a series of proposed cases of thorax computed tomography (CT) and mammography. A total of 26 expert participants from the Radiology Department at Salamanca University Hospital (15 specialists and 11 residents) fulfilled the activity and evaluated different aspects through a series of surveys: software usability, case navigation tools, CAD module utility for learning and JORCAD educational capabilities. Participants also graded imaging cases to establish JORCAD usefulness for training radiology residents. According to the statistical analysis of survey results and expert cases scoring, along with their opinions, it can be concluded that JORCAD software is a useful tool for training future specialists. The combination of CAD with annotated cases from validated databases enhances learning, offering a second opinion and changing the usual training paradigm. Including software as JORCAD in residency training programs of Radiology and other medical specialties would have a positive effect on trainees' background knowledge.

Expert validation of prediction models for a clinical decision-support system in audiology.

For supporting clinical decision-making in audiology, Common Audiological Functional Parameters (CAFPAs) were suggested as an interpretable intermediate representation of audiological information taken from various diagnostic sources within a clinical decision-support system (CDSS). Ten different CAFPAs were proposed to represent specific functional aspects of the human auditory system, namely hearing threshold, supra-threshold deficits, binaural hearing, neural processing, cognitive abilities, and a socio-economic component. CAFPAs were established as a viable basis for deriving audiological findings and treatment recommendations, and it has been demonstrated that model-predicted CAFPAs, with machine learning models trained on expert-labeled patient cases, are sufficiently accurate to be included in a CDSS, but it requires further validation by experts. The present study aimed to validate model-predicted CAFPAs based on previously unlabeled cases from the same data set. Here, we ask to which extent domain experts agree with the model-predicted CAFPAs and whether potential disagreement can be understood in terms of patient characteristics. To these aims, an expert survey was designed and applied to two highly-experienced audiology specialists. They were asked to evaluate model-predicted CAFPAs and estimate audiological findings of the given audiological information about the patients that they were presented with simultaneously. The results revealed strong relative agreement between the two experts and importantly between experts and the prediction for all CAFPAs, except for the neural processing and binaural hearing-related ones. It turned out, however, that experts tend to score CAFPAs in a larger value range, but, on average, across patients with smaller scores as compared with the machine learning models. For the hearing threshold-associated CAFPA in frequencies smaller than 0.75 kHz and the cognitive CAFPA, not only the relative agreement but also the absolute agreement between machine and experts was very high. For those CAFPAs with an average difference between the model- and expert-estimated values, patient characteristics were predictive of the disagreement. The findings are discussed in terms of how they can help toward further improvement of model-predicted CAFPAs to be incorporated in a CDSS for audiology.

Method of preparing a document for survey instrument validation by experts.

Validation of a survey instrument is an important activity in the research process. Face validity and content validity, though being qualitative methods, are essential steps in validating how far the survey instrument can measure what it is intended for. These techniques are used in both scale development processes and a questionnaire that may contain multiple scales. In the face and content validation, a survey instrument is usually validated by experts from academics and practitioners from field or industry. Researchers face challenges in conducting a proper validation because of the lack of an appropriate method for communicating the requirement and receiving the feedback. In this Paper, the authors develop a template that could be used for the validation of survey instrument. In instrument development process, after the item pool is generated, the template is completed and sent to the reviewer. The reviewer will be able to give the necessary feedback through the template that will be helpful to the researcher in improving the instrument.

Woody vegetation dynamics in the tropical and subtropical Andes from 2001 to 2014: Satellite image interpretation and expert validation.

The interactions between climate and land-use change are dictating the distribution of flora and fauna and reshuffling biotic community composition around the world. Tropical mountains are particularly sensitive because they often have a high human population density, a long history of agriculture, range-restricted species, and high-beta diversity due to a steep elevation gradient. Here we evaluated the change in distribution of woody vegetation in the tropical Andes of South America for the period 2001-2014. For the analyses we created annual land-cover/land-use maps using MODIS satellite data at 250 m pixel resolution, calculated the cover of woody vegetation (trees and shrubs) in 9,274 hexagons of 115.47 km2 , and then determined if there was a statistically significant (p < 0.05) 14 year linear trend (positive-forest gain, negative-forest loss) within each hexagon. Of the 1,308 hexagons with significant trends, 36.6% (n = 479) lost forests and 63.4% (n = 829) gained forests. We estimated an overall net gain of ~500,000 ha in woody vegetation. Forest loss dominated the 1,000-1,499 m elevation zone and forest gain dominated above 1,500 m. The most important transitions were forest loss at lower elevations for pastures and croplands, forest gain in abandoned pastures and cropland in mid-elevation areas, and shrub encroachment into highland grasslands. Expert validation confirmed the observed trends, but some areas of apparent forest gain were associated with new shade coffee, pine, or eucalypt plantations. In addition, after controlling for elevation and country, forest gain was associated with a decline in the rural population. Although we document an overall gain in forest cover, the recent reversal of forest gains in Colombia demonstrates that these coupled natural-human systems are highly dynamic and there is an urgent need of a regional real-time land-use, biodiversity, and ecosystem services monitoring network.

Propuesta de instrumento para evaluar la calidad percibida por pacientes tratados por cirugía de mínimo acceso / Proposition of a questionnaire to evaluate the patient-perceived quality on minimal access surgery

RESUMEN Introducción: La cirugía de mínimo acceso ha tenido un desarrollo importante en las últimas dos décadas en Cuba. La satisfacción de la población con estos servicios, de innegable predilección por su efectividad e inocuidad, es cada vez mayor, sin embargo en la actualidad el nivel de satisfacción de los pacientes sobre bases científicas no se conoce. Objetivo: Elaborar y validar por expertos un instrumento para medir la percepción de la calidad de la atención médica que tienen los pacientes operados por cirugía de mínimo acceso. Métodos: Se elaboró y validó por expertos, un instrumento de medición, incluida la encuesta y estándares, para evaluar la calidad de la atención médica percibida por los pacientes operados, en las dimensiones de estructura, proceso y resultado. Resultados: El 100 por ciento de las preguntas fueron consideradas por los expertos justificables y pertinentes en el instrumento. Tres preguntas no cumplieron alguno de los requisitos de Moriyama y fueron modificadas. Fueron incorporados nuevos ítems relacionados fundamentalmente con procesos de atención médica. Se modificó la escala ordinal de Likert de cinco pasos por una de cuatro pasos. Se fijaron los estándares de calidad iguales o superiores al 80 por ciento, para cada uno de los tres niveles: por cada dimensión, por cada criterio para el conjunto de evaluados y por cada paciente según todos los criterios. Conclusión: El estudio aporta un instrumento validado por expertos, para evaluar la satisfacción de los pacientes en el ámbito de la cirugía de mínimo acceso(AU)

ABSTRACT Introduction: In the last two decades Minimal Access Surgery has had an important development in Cuba. Patient satisfaction with these services is higher, nevertheless today it is not well knows on a scientific basis. Objective: design and validate by experts a measurement instrument to assess the patient-perceived quality of patients operated by minimal access surgery. Methods: A measurement instrument, including questionnaire and standards, was developed and validated by experts, to evaluate the patient-perceived quality in the dimensions structure, process and results. Results: The experts considered the one hundred percent of questions in the instrument justified and relevant. Only three questions no fulfilled with one of the Moriyama's requirement and were improved. New items related with processes criteria were included. The ordinal 5-point Likert scale was modified to 4-point scale. Quality standards were fixed to 80 percent or higher for each of three levels: each dimension, each criterion for all evaluated and each patient according to all criteria. Conclusion: The study shows a validated instrument by experts to evaluate the patient's satisfaction in minimal access surgery(AU)

How do experts reporting for the legal process validate symptoms? The results of a survey.

This article examines the views of experts from a range of disciplines and how they view symptoms given to them by claimants in matters of personal injury or medical negligence assessments. The survey was carried out in 2009 and looks at current practice and attitudes from a number of different disciplines. The survey included questions looking at what percentage of cases were thought to be genuine, symptoms most likely to be elaborated, methods for assessing symptom validity, and documentary evidence required for a report. This article highlights the importance of looking at symptom validation in the legal process.