Assessment of Clinical Metadata on the Accuracy of Retinal Fundus Image Labels in Diabetic Retinopathy in Uganda: Case-Crossover Study Using the Multimodal Database of Retinal Images in Africa.

BACKGROUND: Labeling color fundus photos (CFP) is an important step in the development of artificial intelligence screening algorithms for the detection of diabetic retinopathy (DR). Most studies use the International Classification of Diabetic Retinopathy (ICDR) to assign labels to CFP, plus the presence or absence of macular edema (ME). Images can be grouped as referrable or nonreferrable according to these classifications. There is little guidance in the literature about how to collect and use metadata as a part of the CFP labeling process. OBJECTIVE: This study aimed to improve the quality of the Multimodal Database of Retinal Images in Africa (MoDRIA) by determining whether the availability of metadata during the image labeling process influences the accuracy, sensitivity, and specificity of image labels. MoDRIA was developed as one of the inaugural research projects of the Mbarara University Data Science Research Hub, part of the Data Science for Health Discovery and Innovation in Africa (DS-I Africa) initiative. METHODS: This is a crossover assessment with 2 groups and 2 phases. Each group had 10 randomly assigned labelers who provided an ICDR score and the presence or absence of ME for each of the 50 CFP in a test image with and without metadata including blood pressure, visual acuity, glucose, and medical history. Specificity and sensitivity of referable retinopathy were based on ICDR scores, and ME was calculated using a 2-sided t test. Comparison of sensitivity and specificity for ICDR scores and ME with and without metadata for each participant was calculated using the Wilcoxon signed rank test. Statistical significance was set at P<.05. RESULTS: The sensitivity for identifying referrable DR with metadata was 92.8% (95% CI 87.6-98.0) compared with 93.3% (95% CI 87.6-98.9) without metadata, and the specificity was 84.9% (95% CI 75.1-94.6) with metadata compared with 88.2% (95% CI 79.5-96.8) without metadata. The sensitivity for identifying the presence of ME was 64.3% (95% CI 57.6-71.0) with metadata, compared with 63.1% (95% CI 53.4-73.0) without metadata, and the specificity was 86.5% (95% CI 81.4-91.5) with metadata compared with 87.7% (95% CI 83.9-91.5) without metadata. The sensitivity and specificity of the ICDR score and the presence or absence of ME were calculated for each labeler with and without metadata. No findings were statistically significant. CONCLUSIONS: The sensitivity and specificity scores for the detection of referrable DR were slightly better without metadata, but the difference was not statistically significant. We cannot make definitive conclusions about the impact of metadata on the sensitivity and specificity of image labels in our study. Given the importance of metadata in clinical situations, we believe that metadata may benefit labeling quality. A more rigorous study to determine the sensitivity and specificity of CFP labels with and without metadata is recommended.

Advancing healthcare with artificial intelligence: diagnostic accuracy of machine learning algorithm in diagnosis of diabetic retinopathy in the Brazilian population.

BACKGROUND: In healthcare systems in general, access to diabetic retinopathy (DR) screening is limited. Artificial intelligence has the potential to increase care delivery. Therefore, we trained and evaluated the diagnostic accuracy of a machine learning algorithm for automated detection of DR. METHODS: We included color fundus photographs from individuals from 4 databases (primary and specialized care settings), excluding uninterpretable images. The datasets consist of images from Brazilian patients, which differs from previous work. This modification allows for a more tailored application of the model to Brazilian patients, ensuring that the nuances and characteristics of this specific population are adequately captured. The sample was fractionated in training (70%) and testing (30%) samples. A convolutional neural network was trained for image classification. The reference test was the combined decision from three ophthalmologists. The sensitivity, specificity, and area under the ROC curve of the algorithm for detecting referable DR (moderate non-proliferative DR; severe non-proliferative DR; proliferative DR and/or clinically significant macular edema) were estimated. RESULTS: A total of 15,816 images (4590 patients) were included. The overall prevalence of any degree of DR was 26.5%. Compared with human evaluators (manual method of diagnosing DR performed by an ophthalmologist), the deep learning algorithm achieved an area under the ROC curve of 0.98 (95% CI 0.97-0.98), with a specificity of 94.6% (95% CI 93.8-95.3) and a sensitivity of 93.5% (95% CI 92.2-94.9) at the point of greatest efficiency to detect referable DR. CONCLUSIONS: A large database showed that this deep learning algorithm was accurate in detecting referable DR. This finding aids to universal healthcare systems like Brazil, optimizing screening processes and can serve as a tool for improving DR screening, making it more agile and expanding care access.

BRSET: A Brazilian Multilabel Ophthalmological Dataset of Retina Fundus Photos.

INTRODUCTION: The Brazilian Multilabel Ophthalmological Dataset (BRSET) addresses the scarcity of publicly available ophthalmological datasets in Latin America. BRSET comprises 16,266 color fundus retinal photos from 8,524 Brazilian patients, aiming to enhance data representativeness, serving as a research and teaching tool. It contains sociodemographic information, enabling investigations into differential model performance across demographic groups. METHODS: Data from three São Paulo outpatient centers yielded demographic and medical information from electronic records, including nationality, age, sex, clinical history, insulin use, and duration of diabetes diagnosis. A retinal specialist labeled images for anatomical features (optic disc, blood vessels, macula), quality control (focus, illumination, image field, artifacts), and pathologies (e.g., diabetic retinopathy). Diabetic retinopathy was graded using International Clinic Diabetic Retinopathy and Scottish Diabetic Retinopathy Grading. Validation used a ConvNext model trained during 50 epochs using a weighted cross entropy loss to avoid overfitting, with 70% training (20% validation), and 30% testing subsets. Performance metrics included area under the receiver operating curve (AUC) and Macro F1-score. Saliency maps were calculated for interpretability. RESULTS: BRSET comprises 65.1% Canon CR2 and 34.9% Nikon NF5050 images. 61.8% of the patients are female, and the average age is 57.6 (± 18.26) years. Diabetic retinopathy affected 15.8% of patients, across a spectrum of disease severity. Anatomically, 20.2% showed abnormal optic discs, 4.9% abnormal blood vessels, and 28.8% abnormal macula. A ConvNext V2 model was trained and evaluated BRSET in four prediction tasks: "binary diabetic retinopathy diagnosis (Normal vs Diabetic Retinopathy)" (AUC: 97, F1: 89); "3 class diabetic retinopathy diagnosis (Normal, Proliferative, Non-Proliferative)" (AUC: 97, F1: 82); "diabetes diagnosis" (AUC: 91, F1: 83); "sex classification" (AUC: 87, F1: 70). DISCUSSION: BRSET is the first multilabel ophthalmological dataset in Brazil and Latin America. It provides an opportunity for investigating model biases by evaluating performance across demographic groups. The model performance of three prediction tasks demonstrates the value of the dataset for external validation and for teaching medical computer vision to learners in Latin America using locally relevant data sources.

Image quality comparison of AirDoc portable retina camera versus eyer in a diabetic retinopathy screening program.

BACKGROUND: Diabetic retinopathy (DR) stands as the foremost cause of preventable blindness in adults. Despite efforts to expand DR screening coverage in the Brazilian public healthcare system, challenges persist due to various factors including social, medical, and financial constraints. Our objective was to evaluate the quality of images obtained with the AirDoc, a novel device, compared to Eyer portable camera which has already been clinically validated. METHODS: Images were captured by two portable retinal devices: AirDoc and Eyer. The included patients had their fundus images obtained in a screening program conducted in Blumenau, Santa Catarina. Two retina specialists independently assessed image's quality. A comparison was performed between both devices regarding image quality and the presence of artifacts. RESULTS: The analysis included 129 patients (mean age of 61 years), with 29 (43.28%) male and an average disease duration of 11.1 ± 8 years. In Ardoc, 21 (16.28%) images were classified as poor quality, with 88 (68%) presenting artifacts; in Eyer, 4 (3.1%) images were classified as poor quality, with 94 (72.87%) presenting artifacts. CONCLUSIONS: Although both Eyer and AirDoc devices show potential as screening tools, the AirDoc images displayed higher rates of ungradable and low-quality images, that may directly affect the DR and DME grading. We must acknowledge the limitations of our study, including the relatively small sample size. Therefore, the interpretations of our analyses should be approached with caution, and further investigations with larger patient cohorts are warranted to validate our findings.

DF-DM: A foundational process model for multimodal data fusion in the artificial intelligence era.

In the big data era, integrating diverse data modalities poses significant challenges, particularly in complex fields like healthcare. This paper introduces a new process model for multimodal Data Fusion for Data Mining, integrating embeddings and the Cross-Industry Standard Process for Data Mining with the existing Data Fusion Information Group model. Our model aims to decrease computational costs, complexity, and bias while improving efficiency and reliability. We also propose "disentangled dense fusion," a novel embedding fusion method designed to optimize mutual information and facilitate dense inter-modality feature interaction, thereby minimizing redundant information. We demonstrate the model's efficacy through three use cases: predicting diabetic retinopathy using retinal images and patient metadata, domestic violence prediction employing satellite imagery, internet, and census data, and identifying clinical and demographic features from radiography images and clinical notes. The model achieved a Macro F1 score of 0.92 in diabetic retinopathy prediction, an R-squared of 0.854 and sMAPE of 24.868 in domestic violence prediction, and a macro AUC of 0.92 and 0.99 for disease prediction and sex classification, respectively, in radiological analysis. These results underscore the Data Fusion for Data Mining model's potential to significantly impact multimodal data processing, promoting its adoption in diverse, resource-constrained settings.

Automated Identification of Different Severity Levels of Diabetic Retinopathy Using a Handheld Fundus Camera and Single-Image Protocol.

Purpose: To evaluate the performance of artificial intelligence (AI) systems embedded in a mobile, handheld retinal camera, with a single retinal image protocol, in detecting both diabetic retinopathy (DR) and more-than-mild diabetic retinopathy (mtmDR). Design: Multicenter cross-sectional diagnostic study, conducted at 3 diabetes care and eye care facilities. Participants: A total of 327 individuals with diabetes mellitus (type 1 or type 2) underwent a retinal imaging protocol enabling expert reading and automated analysis. Methods: Participants underwent fundus photographs using a portable retinal camera (Phelcom Eyer). The captured images were automatically analyzed by deep learning algorithms retinal alteration score (RAS) and diabetic retinopathy alteration score (DRAS), consisting of convolutional neural networks trained on EyePACS data sets and fine-tuned using data sets of portable device fundus images. The ground truth was the classification of DR corresponding to adjudicated expert reading, performed by 3 certified ophthalmologists. Main Outcome Measures: Primary outcome measures included the sensitivity and specificity of the AI system in detecting DR and/or mtmDR using a single-field, macula-centered fundus photograph for each eye, compared with a rigorous clinical reference standard comprising the reading center grading of 2-field imaging protocol using the International Classification of Diabetic Retinopathy severity scale. Results: Of 327 analyzed patients (mean age, 57.0 ± 16.8 years; mean diabetes duration, 16.3 ± 9.7 years), 307 completed the study protocol. Sensitivity and specificity of the AI system were high in detecting any DR with DRAS (sensitivity, 90.48% [95% confidence interval (CI), 84.99%-94.46%]; specificity, 90.65% [95% CI, 84.54%-94.93%]) and mtmDR with the combination of RAS and DRAS (sensitivity, 90.23% [95% CI, 83.87%-94.69%]; specificity, 85.06% [95% CI, 78.88%-90.00%]). The area under the receiver operating characteristic curve was 0.95 for any DR and 0.89 for mtmDR. Conclusions: This study showed a high accuracy for the detection of DR in different levels of severity with a single retinal photo per eye in an all-in-one solution, composed of a portable retinal camera powered by AI. Such a strategy holds great potential for increasing coverage rates of screening programs, contributing to prevention of avoidable blindness. Financial Disclosures: F.K.M. is a medical consultant for Phelcom Technologies. J.A.S. is Chief Executive Officer and proprietary of Phelcom Technologies. D.L. is Chief Technology Officer and proprietary of Phelcom Technologies. P.V.P. is an employee at Phelcom Technologies.

BOLD: Blood-gas and Oximetry Linked Dataset.

Pulse oximeters measure peripheral arterial oxygen saturation (SpO2) noninvasively, while the gold standard (SaO2) involves arterial blood gas measurement. There are known racial and ethnic disparities in their performance. BOLD is a dataset that aims to underscore the importance of addressing biases in pulse oximetry accuracy, which disproportionately affect darker-skinned patients. The dataset was created by harmonizing three Electronic Health Record databases (MIMIC-III, MIMIC-IV, eICU-CRD) comprising Intensive Care Unit stays of US patients. Paired SpO2 and SaO2 measurements were time-aligned and combined with various other sociodemographic and parameters to provide a detailed representation of each patient. BOLD includes 49,099 paired measurements, within a 5-minute window and with oxygen saturation levels between 70-100%. Minority racial and ethnic groups account for ~25% of the data - a proportion seldom achieved in previous studies. The codebase is publicly available. Given the prevalent use of pulse oximeters in the hospital and at home, we hope that BOLD will be leveraged to develop debiasing algorithms that can result in more equitable healthcare solutions.

Automated machine learning model for fundus image classification by health-care professionals with no coding experience.

To assess the feasibility of code-free deep learning (CFDL) platforms in the prediction of binary outcomes from fundus images in ophthalmology, evaluating two distinct online-based platforms (Google Vertex and Amazon Rekognition), and two distinct datasets. Two publicly available datasets, Messidor-2 and BRSET, were utilized for model development. The Messidor-2 consists of fundus photographs from diabetic patients and the BRSET is a multi-label dataset. The CFDL platforms were used to create deep learning models, with no preprocessing of the images, by a single ophthalmologist without coding expertise. The performance metrics employed to evaluate the models were F1 score, area under curve (AUC), precision and recall. The performance metrics for referable diabetic retinopathy and macular edema were above 0.9 for both tasks and CDFL. The Google Vertex models demonstrated superior performance compared to the Amazon models, with the BRSET dataset achieving the highest accuracy (AUC of 0.994). Multi-classification tasks using only BRSET achieved similar overall performance between platforms, achieving AUC of 0.994 for laterality, 0.942 for age grouping, 0.779 for genetic sex identification, 0.857 for optic, and 0.837 for normality with Google Vertex. The study demonstrates the feasibility of using automated machine learning platforms for predicting binary outcomes from fundus images in ophthalmology. It highlights the high accuracy achieved by the models in some tasks and the potential of CFDL as an entry-friendly platform for ophthalmologists to familiarize themselves with machine learning concepts.

Synchronous Diagnosis of Diabetic Retinopathy by a Handheld Retinal Camera, Artificial Intelligence, and Simultaneous Specialist Confirmation.

PURPOSE: Diabetic retinopathy (DR) is a leading cause of preventable blindness, particularly in underserved regions where access to ophthalmic care is limited. This study presents a proof of concept for utilizing a portable handheld retinal camera with an embedded artificial intelligence (AI) platform, complemented by a synchronous remote confirmation by retina specialists, for DR screening in an underserved rural area. DESIGN: Retrospective cohort study. SUBJECTS: A total of 1115 individuals with diabetes. METHODS: A retrospective analysis of a screening initiative conducted in 4 municipalities in Northeastern Brazil, targeting the diabetic population. A portable handheld retinal camera captured macula-centered and disc-centered images, which were analyzed by the AI system. Immediate push notifications were sent out to retina specialists upon the detection of significant abnormalities, enabling synchronous verification and confirmation, with on-site patient feedback within minutes. Referral criteria were established, and all referred patients underwent a complete ophthalmic work-up and subsequent treatment. MAIN OUTCOME MEASURES: Proof-of-concept implementation success. RESULTS: Out of 2052 invited individuals, 1115 participated, with a mean age of 60.93 years and diabetes duration of 7.52 years; 66.03% were women. The screening covered 2222 eyes, revealing various retinal conditions. Referable eyes for DR were 11.84%, with an additional 13% for other conditions (diagnoses included various stages of DR, media opacity, nevus, drusen, enlarged cup-to-disc ratio, pigmentary changes, and other). Artificial intelligence performance for overall detection of referable cases (both DR and other conditions) was as follows: sensitivity 84.23% (95% confidence interval (CI), 82.63-85.84), specificity 80.79% (95% CI, 79.05-82.53). When we assessed whether AI matched any clinical diagnosis, be it referable or not, sensitivity was 85.67% (95% CI, 84.12-87.22), specificity was 98.86 (95% CI, 98.39-99.33), and area under the curve was 0.92 (95% CI, 0.91-0.94). CONCLUSIONS: The integration of a portable device, AI analysis, and synchronous medical validation has the potential to play a crucial role in preventing blindness from DR, especially in socially unequal scenarios. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Does diversity beget diversity? A scientometric analysis of over 150,000 studies and 49,000 authors published in high-impact medical journals between 2007 and 2022.

Background: Health research that significantly impacts global clinical practice and policy is often published in high-impact factor (IF) medical journals. These outlets play a pivotal role in the worldwide dissemination of novel medical knowledge. However, researchers identifying as women and those affiliated with institutions in low- and middle-income countries (LMIC) have been largely underrepresented in high-IF journals across multiple fields of medicine. To evaluate disparities in gender and geographical representation among authors who have published in any of five top general medical journals, we conducted scientometric analyses using a large-scale dataset extracted from the New England Journal of Medicine (NEJM), Journal of the American Medical Association (JAMA), The British Medical Journal (BMJ), The Lancet, and Nature Medicine. Methods: Author metadata from all articles published in the selected journals between 2007 and 2022 were collected using the DimensionsAI platform. The Genderize.io API was then utilized to infer each author's likely gender based on their extracted first name. The World Bank country classification was used to map countries associated with researcher affiliations to the LMIC or the high-income country (HIC) category. We characterized the overall gender and country income category representation across the medical journals. In addition, we computed article-level diversity metrics and contrasted their distributions across the journals. Findings: We studied 151,536 authors across 49,764 articles published in five top medical journals, over a long period spanning 15 years. On average, approximately one-third (33.1%) of the authors of a given paper were inferred to be women; this result was consistent across the journals we studied. Further, 86.6% of the teams were exclusively composed of HIC authors; in contrast, only 3.9% were exclusively composed of LMIC authors. The probability of serving as the first or last author was significantly higher if the author was inferred to be a man (18.1% vs 16.8%, P < .01) or was affiliated with an institution in a HIC (16.9% vs 15.5%, P < .01). Our primary finding reveals that having a diverse team promotes further diversity, within the same dimension (i.e., gender or geography) and across dimensions. Notably, papers with at least one woman among the authors were more likely to also involve at least two LMIC authors (11.7% versus 10.4% in baseline, P < .001; based on inferred gender); conversely, papers with at least one LMIC author were more likely to also involve at least two women (49.4% versus 37.6%, P < .001; based on inferred gender). Conclusion: We provide a scientometric framework to assess authorship diversity. Our research suggests that the inclusiveness of high-impact medical journals is limited in terms of both gender and geography. We advocate for medical journals to adopt policies and practices that promote greater diversity and collaborative research. In addition, our findings offer a first step towards understanding the composition of teams conducting medical research globally and an opportunity for individual authors to reflect on their own collaborative research practices and possibilities to cultivate more diverse partnerships in their work.

Transition from an in-person to a telemedicine diabetic retinopathy screening program.

PURPOSE: Timely screening and treatment are essential for preventing diabetic retinopathy blindness. Improving screening workflows can reduce waiting times for specialist evaluation and thus enhance patient outcomes. This study assessed different screening approaches in a Brazilian public healthcare setting. METHODS: This retrospective study evaluated a telemedicine-based diabetic retinopathy screening implemented during the COVID-19 pandemic and compared it with in-person strategies. The evaluation was conducted from the perspective of a specialized referral center in an urban area of Central-West Brazil. In the telemedicine approach, a trained technician would capture retinal images by using a handheld camera. These images were sent to specialists for remote evaluation. Patient variables, including age, gender, duration of diabetes diagnosis, diabetes treatment, comorbidities, and waiting time, were analyzed and compared. RESULTS: In total, 437 patients with diabetes mellitus were included in the study (mean age: 62.5 ± 11.0 years, female: 61.7%, mean diabetes duration: 15.3 ± 9.7 years, insulin users: 67.8%). In the in-person assessment group, the average waiting time between primary care referral and specialist evaluation was 292.3 ± 213.9 days, and the referral rate was 73.29%. In the telemedicine group, the average waiting time was 158.8 ± 192.4 days, and the referral rate was 29.38%. The telemedicine approach significantly reduced the waiting time (p<0.001) and significantly lowered the referral rate (p<0.001). CONCLUSION: The telemedicine approach significantly reduced the waiting time for specialist evaluation in a real-world setting. Employing portable retinal cameras may address the burden of diabetic retinopathy, especially in resource-limited settings.

Ophthalmology Optical Coherence Tomography Databases for Artificial Intelligence Algorithm: A Review.

BACKGROUND: Imaging plays a pivotal role in eye assessment. With the introduction of advanced machine learning and artificial intelligence (AI), the focus has shifted to imaging datasets in ophthalmology. While disparities and health inequalities hidden within data are well-documented, the ophthalmology field faces specific challenges to the creation and maintenance of datasets. Optical Coherence Tomography (OCT) is useful for the diagnosis and monitoring of retinal pathologies, making it valuable for AI applications. This review aims to identify and compare the landscape of publicly available optical coherence tomography databases for AI applications. METHODS: We conducted a literature review on OCT and AI articles with publicly accessible datasets, using PubMed, Scopus, and Web of Science databases. The review retrieved 183 articles, and after full-text analysis, 50 articles were included. From the included articles were identified 8 publicly available OCT datasets, focusing on patient demographics and clinical details for thorough assessment and comparison. RESULTS: The resulting datasets encompass 154,313 images collected from Spectralis, Cirrus HD, Topcon 3D, and Bioptigen devices. These datasets included normal exams, age-related macular degeneration, and diabetic maculopathy, among others. Comprehensive demographic information is available in one dataset and the USA is the most represented population. DISCUSSION: Current publicly available OCT databases for AI applications exhibit limitations, stemming from their non-representative nature and the lack of comprehensive demographic information. Limited datasets hamper research and equitable AI development. To promote equitable AI algorithmic development in ophthalmology, there is a need for the creation and dissemination of more representative datasets.

BRSET: A Brazilian Multilabel Ophthalmological Dataset of Retina Fundus Photos.

Introduction: The Brazilian Multilabel Ophthalmological Dataset (BRSET) addresses the scarcity of publicly available ophthalmological datasets in Latin America. BRSET comprises 16,266 color fundus retinal photos from 8,524 Brazilian patients, aiming to enhance data representativeness, serving as a research and teaching tool. It contains sociodemographic information, enabling investigations into differential model performance across demographic groups. Methods: Data from three São Paulo outpatient centers yielded demographic and medical information from electronic records, including nationality, age, sex, clinical history, insulin use, and duration of diabetes diagnosis. A retinal specialist labeled images for anatomical features (optic disc, blood vessels, macula), quality control (focus, illumination, image field, artifacts), and pathologies (e.g., diabetic retinopathy). Diabetic retinopathy was graded using International Clinic Diabetic Retinopathy and Scottish Diabetic Retinopathy Grading. Validation used Dino V2 Base for feature extraction, with 70% training and 30% testing subsets. Support Vector Machines (SVM) and Logistic Regression (LR) were employed with weighted training. Performance metrics included area under the receiver operating curve (AUC) and Macro F1-score. Results: BRSET comprises 65.1% Canon CR2 and 34.9% Nikon NF5050 images. 61.8% of the patients are female, and the average age is 57.6 years. Diabetic retinopathy affected 15.8% of patients, across a spectrum of disease severity. Anatomically, 20.2% showed abnormal optic discs, 4.9% abnormal blood vessels, and 28.8% abnormal macula. Models were trained on BRSET in three prediction tasks: "diabetes diagnosis"; "sex classification"; and "diabetic retinopathy diagnosis". Discussion: BRSET is the first multilabel ophthalmological dataset in Brazil and Latin America. It provides an opportunity for investigating model biases by evaluating performance across demographic groups. The model performance of three prediction tasks demonstrates the value of the dataset for external validation and for teaching medical computer vision to learners in Latin America using locally relevant data sources.

Image quality assessment of retinal fundus photographs for diabetic retinopathy in the machine learning era: a review.

This study aimed to evaluate the image quality assessment (IQA) and quality criteria employed in publicly available datasets for diabetic retinopathy (DR). A literature search strategy was used to identify relevant datasets, and 20 datasets were included in the analysis. Out of these, 12 datasets mentioned performing IQA, but only eight specified the quality criteria used. The reported quality criteria varied widely across datasets, and accessing the information was often challenging. The findings highlight the importance of IQA for AI model development while emphasizing the need for clear and accessible reporting of IQA information. The study suggests that automated quality assessments can be a valid alternative to manual labeling and emphasizes the importance of establishing quality standards based on population characteristics, clinical use, and research purposes. In conclusion, image quality assessment is important for AI model development; however, strict data quality standards must not limit data sharing. Given the importance of IQA for developing, validating, and implementing deep learning (DL) algorithms, it's recommended that this information be reported in a clear, specific, and accessible way whenever possible. Automated quality assessments are a valid alternative to the traditional manual labeling process, and quality standards should be determined according to population characteristics, clinical use, and research purpose.

An emergency room influx and trauma cases prediction in a Brazilian ophthalmological hospital by an ophthalmologist without code experience / Predição do influxo de atendimentos e casos de trauma em um serviço de urgência oftalmológico brasileiro realizado por um oftalmologista sem experiência em programação

ABSTRACT Purpose: The emergency medical service is a fundamental part of healthcare, albeit crowded emergency rooms lead to delayed and low-quality assistance in actual urgent cases. Machine-learning algorithms can provide a smart and effective estimation of emergency patients' volume, which was previously restricted to artificial intelligence (AI) experts in coding and computer science but is now feasible by anyone without any coding experience through auto machine learning. This study aimed to create a machine-learning model designed by an ophthalmologist without any coding experience using AutoML to predict the influx in the emergency department and trauma cases. Methods: A dataset of 356,611 visits at Hospital da Universidade Federal de São Paulo from January 01, 2014 to December 31, 2019 was included in the model training, which included visits/day and the international classification disease code. The training and prediction were made with the Amazon Forecast by 2 ophthalmologists with no prior coding experience. Results: The forecast period predicted a mean emergency patient volume of 216.27/day in p90, 180.75/day in p50, and 140.35/day in p10, and a mean of 7.42 trauma cases/ day in p90, 3.99/day in p50, and 0.56/day in p10. In January of 2020, there were a total of 6,604 patient visits and a mean of 206.37 patients/day, which is 13.5% less than the p50 prediction. This period involved a total of 199 trauma cases and a mean of 6.21 cases/day, which is 55.77% more traumas than that by the p50 prediction. Conclusions: The development of models was previously restricted to data scientists' experts in coding and computer science, but transfer learning autoML has enabled AI development by any person with no code experience mandatory. This study model showed a close value to the actual 2020 January visits, and the only factors that may have influenced the results between the two approaches are holidays and dataset size. This is the first study to apply AutoML in hospital visits forecast, showing a close prediction of the actual hospital influx.

RESUMO Objetivo: Esse estudo tem como objetivo criar um modelo de Machine Learning por um oftalmologista sem experiência em programação utilizando auto Machine Learning predizendo influxo de pacientes em serviço de emergência e casos de trauma. Métodos: Um dataset de 366,610 visitas em Hospital Universitário da Universidade Federal de São Paulo de 01 de janeiro de 2014 até 31 de dezembro de 2019 foi incluído no treinamento do modelo, incluindo visitas/dia e código internacional de doenças. O treinamento e predição foram realizados com o Amazon Forecast por dois oftalmologistas sem experiência com programação. Resultados: O período de previsão estimou um volume de 206,37 pacientes/dia em p90, 180,75 em p50, 140,35 em p10 e média de 7,42 casos de trauma/dia em p90, 3,99 em p50 e 0,56 em p10. Janeiro de 2020 teve um total de 6.604 pacientes e média de 206,37 pacientes/dia, 13,5% menos do que a predição em p50. O período teve um total de 199 casos de trauma e média de 6,21 casos/dia, 55,77% mais casos do que a predição em p50. Conclusão: O desenvolvimento de modelos era restrito a cientistas de dados com experiencia em programação, porém a transferência de ensino com a tecnologia de auto Machine Learning permite o desenvolvimento de algoritmos por qualquer pessoa sem experiencia em programação. Esse estudo mostra um modelo com valores preditos próximos ao que ocorreram em janeiro de 2020. Fatores que podem ter influenciados no resultado foram feriados e tamanho do banco de dados. Esse é o primeiro estudo que aplicada auto Machine Learning em predição de visitas hospitalares com resultados próximos aos que ocorreram.

Transition from an in-person to a telemedicine diabetic retinopathy screening program

ABSTRACT Purpose: Timely screening and treatment are essential for preventing diabetic retinopathy blindness. Improving screening workflows can reduce waiting times for specialist evaluation and thus enhance patient outcomes. This study assessed different screening approaches in a Brazilian public healthcare setting. Methods: This retrospective study evaluated a telemedicine-based diabetic retinopathy screening implemented during the COVID-19 pandemic and compared it with in-person strategies. The evaluation was conducted from the perspective of a specialized referral center in an urban area of Central-West Brazil. In the telemedicine approach, a trained technician would capture retinal images by using a handheld camera. These images were sent to specialists for remote evaluation. Patient variables, including age, gender, duration of diabetes diagnosis, diabetes treatment, comorbidities, and waiting time, were analyzed and compared. Results: In total, 437 patients with diabetes mellitus were included in the study (mean age: 62.5 ± 11.0 years, female: 61.7%, mean diabetes duration: 15.3 ± 9.7 years, insulin users: 67.8%). In the in-person assessment group, the average waiting time between primary care referral and specialist evaluation was 292.3 ± 213.9 days, and the referral rate was 73.29%. In the telemedicine group, the average waiting time was 158.8 ± 192.4 days, and the referral rate was 29.38%. The telemedicine approach significantly reduced the waiting time (p<0.001) and significantly lowered the referral rate (p<0.001). Conclusion: The telemedicine approach significantly reduced the waiting time for specialist evaluation in a real-world setting. Employing portable retinal cameras may address the burden of diabetic retinopathy, especially in resource-limited settings.

Disparity dashboards: an evaluation of the literature and framework for health equity improvement.

The growing recognition of differences in health outcomes across populations has led to a slow but increasing shift towards transparent reporting of patient outcomes. In addition, pay-for-equity initiatives, such as those proposed by the Centers for Medicare and Medicaid, will require the reporting of health outcomes across subgroups over time. Dashboards offer one means of visualising data in the health-care context that can highlight essential disparities in clinical outcomes, guide targeted quality-improvement efforts, and ultimately improve health equity. In this Viewpoint, we evaluate all studies that have reported the successful development of a disparity dashboard and share the data collected and unintended consequences reported. We propose a framework for systematic equality improvement through incentivisation of the collecting and reporting of health data and through implementation of reward systems to reduce health disparities.

Digital Determinants of Health: Health data poverty amplifies existing health disparities-A scoping review.

Artificial intelligence (AI) and machine learning (ML) have an immense potential to transform healthcare as already demonstrated in various medical specialties. This scoping review focuses on the factors that influence health data poverty, by conducting a literature review, analysis, and appraisal of results. Health data poverty is often an unseen factor which leads to perpetuating or exacerbating health disparities. Improvements or failures in addressing health data poverty will directly impact the effectiveness of AI/ML systems. The potential causes are complex and may enter anywhere along the development process. The initial results highlighted studies with common themes of health disparities (72%), AL/ML bias (28%) and biases in input data (18%). To properly evaluate disparities that exist we recommend a strengthened effort to generate unbiased equitable data, improved understanding of the limitations of AI/ML tools, and rigorous regulation with continuous monitoring of the clinical outcomes of deployed tools.

Performance of ChatGPT-4 in answering questions from the Brazilian National Examination for Medical Degree Revalidation.

OBJECTIVE: The aim of this study was to evaluate the performance of ChatGPT-4.0 in answering the 2022 Brazilian National Examination for Medical Degree Revalidation (Revalida) and as a tool to provide feedback on the quality of the examination. METHODS: A total of two independent physicians entered all examination questions into ChatGPT-4.0. After comparing the outputs with the test solutions, they classified the large language model answers as adequate, inadequate, or indeterminate. In cases of disagreement, they adjudicated and achieved a consensus decision on the ChatGPT accuracy. The performance across medical themes and nullified questions was compared using chi-square statistical analysis. RESULTS: In the Revalida examination, ChatGPT-4.0 answered 71 (87.7%) questions correctly and 10 (12.3%) incorrectly. There was no statistically significant difference in the proportions of correct answers among different medical themes (p=0.4886). The artificial intelligence model had a lower accuracy of 71.4% in nullified questions, with no statistical difference (p=0.241) between non-nullified and nullified groups. CONCLUSION: ChatGPT-4.0 showed satisfactory performance for the 2022 Brazilian National Examination for Medical Degree Revalidation. The large language model exhibited worse performance on subjective questions and public healthcare themes. The results of this study suggested that the overall quality of the Revalida examination questions is satisfactory and corroborates the nullified questions.

BOLD: Blood-gas and Oximetry Linked Dataset - Open Source Research.

Pulse oximeters measure peripheral arterial oxygen saturation (SpO 2 ) noninvasively, while the gold standard (SaO 2 ) involves arterial blood gas measurement. There are known racial and ethnic disparities in their performance. BOLD is a new comprehensive dataset that aims to underscore the importance of addressing biases in pulse oximetry accuracy, which disproportionately affect darker-skinned patients. The dataset was created by harmonizing three Electronic Health Record databases (MIMIC-III, MIMIC-IV, eICU-CRD) comprising Intensive Care Unit stays of US patients. Paired SpO 2 and SaO 2 measurements were time-aligned and combined with various other sociodemographic and parameters to provide a detailed representation of each patient. BOLD includes 49,099 paired measurements, within a 5-minute window and with oxygen saturation levels between 70-100%. Minority racial and ethnic groups account for ∼25% of the data - a proportion seldom achieved in previous studies. The codebase is publicly available. Given the prevalent use of pulse oximeters in the hospital and at home, we hope that BOLD will be leveraged to develop debiasing algorithms that can result in more equitable healthcare solutions.