Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residents.

Background: Artificial intelligence (AI) has been proven useful for the assessment of tubes and lines on chest radiographs of general patients. However, validation on intensive care unit (ICU) patients remains imperative. Methods: This retrospective case-control study evaluated the performance of deep learning (DL) models for tubes and lines classification on both an external public dataset and a local dataset comprising 303 films randomly sampled from the ICU database. The endotracheal tubes (ETTs), central venous catheters (CVCs), and nasogastric tubes (NGTs) were classified into "Normal," "Abnormal," or "Borderline" positions by DL models with and without rule-based modification. Their performance was evaluated using an experienced radiologist as the standard reference. Results: The algorithm showed decreased performance on the local ICU dataset, compared to that of the external dataset, decreasing from the Area Under the Curve of Receiver (AUC) of 0.967 (95 % CI 0.965-0.973) to the AUC of 0.70 (95 % CI 0.68-0.77). Significant improvement in the ETT classification task was observed after modifications were made to the model to allow the use of the spatial relationship between line tips and reference anatomy with the improvement of the AUC, increasing from 0.71 (95 % CI 0.70 - 0.75) to 0.86 (95 % CI 0.83 - 0.94). Conclusions: The externally trained model exhibited limited generalizability on the local ICU dataset. Therefore, evaluating the performance of externally trained AI before integrating it into critical care routine is crucial. Rule-based algorithm may be used in combination with DL to improve results.

Discovery of generalizable TBI phenotypes using multivariate time-series clustering.

Traumatic Brain Injury (TBI) presents a broad spectrum of clinical presentations and outcomes due to its inherent heterogeneity, leading to diverse recovery trajectories and varied therapeutic responses. While many studies have delved into TBI phenotyping for distinct patient populations, identifying TBI phenotypes that consistently generalize across various settings and populations remains a critical research gap. Our research addresses this by employing multivariate time-series clustering to unveil TBI's dynamic intricates. Utilizing a self-supervised learning-based approach to clustering multivariate time-Series data with missing values (SLAC-Time), we analyzed both the research-centric TRACK-TBI and the real-world MIMIC-IV datasets. Remarkably, the optimal hyperparameters of SLAC-Time and the ideal number of clusters remained consistent across these datasets, underscoring SLAC-Time's stability across heterogeneous datasets. Our analysis revealed three generalizable TBI phenotypes (α, ß, and γ), each exhibiting distinct non-temporal features during emergency department visits, and temporal feature profiles throughout ICU stays. Specifically, phenotype α represents mild TBI with a remarkably consistent clinical presentation. In contrast, phenotype ß signifies severe TBI with diverse clinical manifestations, and phenotype γ represents a moderate TBI profile in terms of severity and clinical diversity. Age is a significant determinant of TBI outcomes, with older cohorts recording higher mortality rates. Importantly, while certain features varied by age, the core characteristics of TBI manifestations tied to each phenotype remain consistent across diverse populations.

Scoping review on natural language processing applications in counselling and psychotherapy.

Recent years have witnessed some rapid and tremendous progress in natural language processing (NLP) techniques that are used to analyse text data. This study endeavours to offer an up-to-date review of NLP applications by examining their use in counselling and psychotherapy from 1990 to 2021. The purpose of this scoping review is to identify trends, advancements, challenges and limitations of these applications. Among the 41 papers included in this review, 4 primary study purposes were identified: (1) developing automated coding; (2) predicting outcomes; (3) monitoring counselling sessions; and (4) investigating language patterns. Our findings showed a growing trend in the number of papers utilizing advanced machine learning methods, particularly neural networks. Unfortunately, only a third of the articles addressed the issues of bias and generalizability. Our findings provided a timely systematic update, shedding light on concerns related to bias, generalizability and validity in the context of NLP applications in counselling and psychotherapy.

Generalizable diagnosis of chest radiographs through attention-guided decomposition of images utilizing self-consistency loss.

BACKGROUND: Chest X-ray (CXR) is one of the most commonly performed imaging tests worldwide. Due to its wide usage, there is a growing need for automated and generalizable methods to accurately diagnose these images. Traditional methods for chest X-ray analysis often struggle with generalization across diverse datasets due to variations in imaging protocols, patient demographics, and the presence of overlapping anatomical structures. Therefore, there is a significant demand for advanced diagnostic tools that can consistently identify abnormalities across different patient populations and imaging settings. We propose a method that can provide a generalizable diagnosis of chest X-ray. METHOD: Our method utilizes an attention-guided decomposer network (ADSC) to extract disease maps from chest X-ray images. The ADSC employs one encoder and multiple decoders, incorporating a novel self-consistency loss to ensure consistent functionality across its modules. The attention-guided encoder captures salient features of abnormalities, while three distinct decoders generate a normal synthesized image, a disease map, and a reconstructed input image, respectively. A discriminator differentiates the real and the synthesized normal chest X-rays, enhancing the quality of generated images. The disease map along with the original chest X-ray image are fed to a DenseNet-121 classifier modified for multi-class classification of the input X-ray. RESULTS: Experimental results on multiple publicly available datasets demonstrate the effectiveness of our approach. For multi-class classification, we achieve up to a 3% improvement in AUROC score for certain abnormalities compared to the existing methods. For binary classification (normal versus abnormal), our method surpasses existing approaches across various datasets. In terms of generalizability, we train our model on one dataset and tested it on multiple datasets. The standard deviation of AUROC scores for different test datasets is calculated to measure the variability of performance across datasets. Our model exhibits superior generalization across datasets from diverse sources. CONCLUSIONS: Our model shows promising results for the generalizable diagnosis of chest X-rays. The impacts of using the attention mechanism and the self-consistency loss in our method are evident from the results. In the future, we plan to incorporate Explainable AI techniques to provide explanations for model decisions. Additionally, we aim to design data augmentation techniques to reduce class imbalance in our model.

Uncertainty-aware multiple-instance learning for reliable classification: Application to optical coherence tomography.

Deep learning classification models for medical image analysis often perform well on data from scanners that were used to acquire the training data. However, when these models are applied to data from different vendors, their performance tends to drop substantially. Artifacts that only occur within scans from specific scanners are major causes of this poor generalizability. We aimed to enhance the reliability of deep learning classification models using a novel method called Uncertainty-Based Instance eXclusion (UBIX). UBIX is an inference-time module that can be employed in multiple-instance learning (MIL) settings. MIL is a paradigm in which instances (generally crops or slices) of a bag (generally an image) contribute towards a bag-level output. Instead of assuming equal contribution of all instances to the bag-level output, UBIX detects instances corrupted due to local artifacts on-the-fly using uncertainty estimation, reducing or fully ignoring their contributions before MIL pooling. In our experiments, instances are 2D slices and bags are volumetric images, but alternative definitions are also possible. Although UBIX is generally applicable to diverse classification tasks, we focused on the staging of age-related macular degeneration in optical coherence tomography. Our models were trained on data from a single scanner and tested on external datasets from different vendors, which included vendor-specific artifacts. UBIX showed reliable behavior, with a slight decrease in performance (a decrease of the quadratic weighted kappa (κw) from 0.861 to 0.708), when applied to images from different vendors containing artifacts; while a state-of-the-art 3D neural network without UBIX suffered from a significant detriment of performance (κw from 0.852 to 0.084) on the same test set. We showed that instances with unseen artifacts can be identified with OOD detection. UBIX can reduce their contribution to the bag-level predictions, improving reliability without retraining on new data. This potentially increases the applicability of artificial intelligence models to data from other scanners than the ones for which they were developed. The source code for UBIX, including trained model weights, is publicly available through https://github.com/qurAI-amsterdam/ubix-for-reliable-classification.

Estimating dementia incidence in insured older Asian Americans and Pacific Islanders in California: an application of inverse odds of selection weights.

Literature shows heterogeneous age-standardized dementia incidence rates across US Asian American, Native Hawaiian, and Pacific Islanders (AANHPI), but no estimates of population-representative dementia incidence exist due to lack of AANHPI longitudinal probability samples. We compared harmonized characteristics between AANHPI Kaiser Permanente Northern California members (KPNC cohort) and the target population of AANHPI 60+ with private or Medicare insurance using the California Health Interview Survey. We used stabilized inverse odds of selection weights (sIOSW) to estimate ethnicity-specific crude and age-standardized dementia incidence rates and cumulative risk by age 90 in the target population. Differences between the KPNC cohort and target population varied by ethnicity. sIOSW eliminated most differences in larger ethnic groups; some differences remained in smaller groups. Estimated crude dementia incidence rates using sIOSW (versus unweighted) were similar in Chinese, Filipinos, Pacific Islanders and Vietnamese, and higher in Japanese, Koreans, and South Asians. Unweighted and weighted age-standardized incidence rates differed for South Asians. Unweighted and weighted cumulative risk were similar for all groups. We estimated the first population-representative dementia incidence rates and cumulative risk in AANHPI ethnic groups. We encountered some estimation problems and weighted estimates were imprecise, highlighting challenges using weighting to extend inferences to target populations.

The Factor Structure and Generalizability of the Iranian Socioeconomic Status (SES) Questionnaire Administered in a Nationally Divergent Population.

Background: Measuring socioeconomic status (SES) as an independent variable is challenging, especially in epidemiological and social studies. This issue is more critical in large-scale studies on the national level. The present study aimed to extensively evaluate the validity and reliability of the Iranian SES questionnaire. Methods: This psychometric, cross-sectional study was conducted on 3000 households, selected via random cluster sampling from various areas in East Azerbaijan province and Tehran, Iran. Moreover, 250 students from Tabriz University of Medical Sciences were selected as interviewers to collect data from 40 districts in Iran. The construct validity and internal consistency of the SES questionnaire were assessed using exploratory and confirmatory factor analyses and the Cronbach's alpha. Data analysis was performed in SPSS and AMOS. Results: The complete Iranian version of the SES questionnaire consists of 5 factors. The Cronbach's alpha was calculated to be 0.79, 0.94, 0.66, 0.69, and 0.48 for the occupation, self-evaluation of economic capacity, house and furniture, wealth, and health expenditure, respectively. In addition, the confirmatory factor analysis results indicated the data's compatibility with the 5-factor model (comparative fit index = 0.96; goodness of fit index = 0.95; incremental fit index = 0.96; root mean square error of approximation = 0.05). Conclusion: According to the results, the confirmed validity and reliability of the tool indicated that the Iranian version of the SES questionnaire could be utilized with the same structure on an extensive level and could be applicable for measuring the SES in a broader range of populations.

Eligibility of Asian and European registry patients for phase III trials in heart failure with reduced ejection fraction.

AIMS: Traditional approaches to designing clinical trials for heart failure (HF) have historically relied on expertise and past practices. However, the evolving landscape of healthcare, marked by the advent of novel data science applications and increased data availability, offers a compelling opportunity to transition towards a data-driven paradigm in trial design. This research aims to evaluate the scope and determinants of disparities between clinical trials and registries by leveraging natural language processing for the analysis of trial eligibility criteria. The findings contribute to the establishment of a robust design framework for guiding future HF trials. METHODS AND RESULTS: Interventional phase III trials registered for HF on ClinicalTrials.gov as of the end of 2021 were identified. Natural language processing was used to extract and structure the eligibility criteria for quantitative analysis. The most common criteria for HF with reduced ejection fraction (HFrEF) were applied to estimate patient eligibility as a proportion of registry patients in the ASIAN-HF (N = 4868) and BIOSTAT-CHF registries (N = 2545). Of the 375 phase III trials for HF, 163 HFrEF trials were identified. In these trials, the most frequently encountered inclusion criteria were New York Heart Association (NYHA) functional class (69%), worsening HF (23%), and natriuretic peptides (18%), whereas the most frequent comorbidity-based exclusion criteria were acute coronary syndrome (64%), renal disease (55%), and valvular heart disease (47%). On average, 20% of registry patients were eligible for HFrEF trials. Eligibility distributions did not differ (P = 0.18) between Asian [median eligibility 0.20, interquartile range (IQR) 0.08-0.43] and European registry populations (median 0.17, IQR 0.06-0.39). With time, HFrEF trials became more restrictive, where patient eligibility declined from 0.40 in 1985-2005 to 0.19 in 2016-2022 (P = 0.03). When frequency among trials is taken into consideration, the eligibility criteria that were most restrictive were prior myocardial infarction, NYHA class, age, and prior HF hospitalization. CONCLUSIONS: Based on 14 trial criteria, only one-fifth of registry patients were eligible for phase III HFrEF trials. Overall eligibility rates did not differ between the Asian and European patient cohorts.

Prevalence Increases as Treatments Improve: An Evolutionary Perspective on the Treatment-Prevalence Paradox in Depression.

Depression is an eminently treatable disorder that responds to psychotherapy or medications; the efficacy of each has been established in hundreds of controlled trials. Nonetheless, the prevalence of depression has increased in recent years despite the existence of efficacious treatments-a phenomenon known as the treatment-prevalence paradox. We consider several possible explanations for this paradox, which range from a misunderstanding of the very nature of depression, inflated efficacy of the established treatments, and a lack of access to efficacious delivery of treatments. We find support for each of these possible explanations but especially the notion that large segments of the population lack access to efficacious treatments that are implemented as intended. We conclude by describing the potential of using lay therapists and digital technologies to overcome this lack of access and to reach historically underserved populations and simultaneously guarantee the quality of the interventions delivered.

Reliability of the Test of Gross Motor Development Third Edition Among Children with Developmental Coordination Disorder.

AIM: The Test of Gross Motor Development Third Edition (TGMD-3) is used to assess the development of fundamental movement skills in children from 3 to 10 years old. This study aimed to evaluate the intra-rater, inter-rater, and test-retest reliability and to determine the minimal detectable change (MDC) value of the TGMD-3 in children with developmental coordination disorder (DCD). METHODS: The TGMD-3 was administered to 20 children with DCD. The child's fundamental movement skills were recorded using a digital video camera. Reliability was assessed at two occasions by three raters using the generalizability theory. RESULTS: The TGMD-3 demonstrates good inter-rater reliability for the locomotor skills subscale, the ball skills subscale, and the total score (φ = 0.77 - 0.91), while the intra-rater reliability was even higher (φ = 0.94 - 0.97). Test-retest reliability was also shown to be good (φ = 0.79-0.93). The MDC95 was determined to be 10 points. CONCLUSION: This study provides evidence that the TGMD-3 is a reliable test when used to evaluate fundamental movement skills in children with DCD and suggests that an increase of 10 points represents a significant change in the motor function of a child with DCD.

Heterogeneity in effect size estimates.

A typical empirical study involves choosing a sample, a research design, and an analysis path. Variation in such choices across studies leads to heterogeneity in results that introduce an additional layer of uncertainty, limiting the generalizability of published scientific findings. We provide a framework for studying heterogeneity in the social sciences and divide heterogeneity into population, design, and analytical heterogeneity. Our framework suggests that after accounting for heterogeneity, the probability that the tested hypothesis is true for the average population, design, and analysis path can be much lower than implied by nominal error rates of statistically significant individual studies. We estimate each type's heterogeneity from 70 multilab replication studies, 11 prospective meta-analyses of studies employing different experimental designs, and 5 multianalyst studies. In our data, population heterogeneity tends to be relatively small, whereas design and analytical heterogeneity are large. Our results should, however, be interpreted cautiously due to the limited number of studies and the large uncertainty in the heterogeneity estimates. We discuss several ways to parse and account for heterogeneity in the context of different methodologies.

Integrative nomogram model based on anoikis-related genes enhances prognostic evaluation in colorectal cancer.

Background: Revealing the role of anoikis resistance plays in CRC is significant for CRC diagnosis and treatment. This study integrated the CRC anoikis-related key genes (CRC-AKGs) and established a novel model for improving the efficiency and accuracy of the prognostic evaluation of CRC. Methods: CRC-ARGs were screened out by performing differential expression and univariate Cox analysis. CRC-AKGs were obtained through the LASSO machine learning algorithm and the LASSO Risk-Score was constructed to build a nomogram clinical prediction model combined with the clinical predictors. In parallel, this work developed a web-based dynamic nomogram to facilitate the generalization and practical application of our model. Results: We identified 10 CRC-AKGs and a risk-related prognostic Risk-Score was calculated. Multivariate COX regression analysis indicated that the Risk-Score, TNM stage, and age were independent risk factors that significantly associated with the CRC prognosis(p < 0.05). A prognostic model was built to predict the outcome with satisfied accuracy (3-year AUC = 0.815) for CRC individuals. The web interactive nomogram (https://yuexiaozhang.shinyapps.io/anoikisCRC/) showed strong generalizability of our model. In parallel, a substantial correlation between tumor microenvironment and Risk-Score was discovered in the present work. Conclusion: This study reveals the potential role of anoikis in CRC and sets new insights into clinical decision-making in colorectal cancer based on both clinical and sequencing data. Also, the interactive tool provides researchers with a user-friendly interface to input relevant clinical variables and obtain personalized risk predictions or prognostic assessments based on our established model.

Customizing Bayesian multivariate generalizability theory to mixed-format tests.

Mixed-format tests, which typically include dichotomous items and polytomously scored tasks, are employed to assess a wider range of knowledge and skills. Recent behavioral and educational studies have highlighted their practical importance and methodological developments, particularly within the context of multivariate generalizability theory. However, the diverse response types and complex designs of these tests pose significant analytical challenges when modeling data simultaneously. Current methods often struggle to yield reliable results, either due to the inappropriate treatment of different types of response data separately or the imposition of identical covariates across various response types. Moreover, there are few software packages or programs that offer customized solutions for modeling mixed-format tests, addressing these limitations. This tutorial provides a detailed example of using a Bayesian approach to model data collected from a mixed-format test, comprising multiple-choice questions and free-response tasks. The modeling was conducted using the Stan software within the R programming system, with Stan codes tailored to the structure of the test design, following the principles of multivariate generalizability theory. By further examining the effects of prior distributions in this example, this study demonstrates how the adaptability of Bayesian models to diverse test formats, coupled with their potential for nuanced analysis, can significantly advance the field of psychometric modeling.

Selection Bias in Health Research: Quantifying, Eliminating, or Exacerbating Health Disparities?

Purpose of review: To summarize recent literature on selection bias in disparities research addressing either descriptive or causal questions, with examples from dementia research. Recent findings: Defining a clear estimand, including the target population, is essential to assess whether generalizability bias or collider-stratification bias are threats to inferences. Selection bias in disparities research can result from sampling strategies, differential inclusion pipelines, loss to follow-up, and competing events. If competing events occur, several potentially relevant estimands can be estimated under different assumptions, with different interpretations. The apparent magnitude of a disparity can differ substantially based on the chosen estimand. Both randomized and observational studies may misrepresent health disparities or heterogeneity in treatment effects if they are not based on a known sampling scheme. Conclusion: Researchers have recently made substantial progress in conceptualization and methods related to selection bias. This progress will improve the relevance of both descriptive and causal health disparities research.

Cognition of Time and Thinking Beyond.

A common research protocol in cognitive neuroscience is to train subjects to perform deliberately designed experiments while recording brain activity, with the aim of understanding the brain mechanisms underlying cognition. However, how the results of this protocol of research can be applied in technology is seldom discussed. Here, I review the studies on time processing of the brain as examples of this research protocol, as well as two main application areas of neuroscience (neuroengineering and brain-inspired artificial intelligence). Time processing is a fundamental dimension of cognition, and time is also an indispensable dimension of any real-world signal to be processed in technology. Therefore, one may expect that the studies of time processing in cognition profoundly influence brain-related technology. Surprisingly, I found that the results from cognitive studies on timing processing are hardly helpful in solving practical problems. This awkward situation may be due to the lack of generalizability of the results of cognitive studies, which are under well-controlled laboratory conditions, to real-life situations. This lack of generalizability may be rooted in the fundamental unknowability of the world (including cognition). Overall, this paper questions and criticizes the usefulness and prospect of the abovementioned research protocol of cognitive neuroscience. I then give three suggestions for future research. First, to improve the generalizability of research, it is better to study brain activity under real-life conditions instead of in well-controlled laboratory experiments. Second, to overcome the unknowability of the world, we can engineer an easily accessible surrogate of the object under investigation, so that we can predict the behavior of the object under investigation by experimenting on the surrogate. Third, the paper calls for technology-oriented research, with the aim of technology creation instead of knowledge discovery.

How Generalizable Are Findings from a Community-Based Prospective Cohort Study? Extending Estimates from the Adult Changes in Thought Study to Its Source Population.

Background: The Adult Changes in Thought (ACT) study is a cohort of Kaiser Permanente Washington members ages 65+ that began in 1994. Objective: We wanted to know how well ACT participants represented all older adults in the region, and how well ACT findings on eye disease and its relationship with Alzheimer's disease generalized to all older adults in the Seattle Metropolitan Region. Methods: We used participation weights derived from pooling ACT and Behavioral Risk Factor Surveillance System (BRFSS) data to estimate prevalences of common eye diseases and their associations with Alzheimer's disease incidence. Cox proportional hazards models accounted for age, education, smoking, sex, and APOE genotype. Confidence intervals for weighted analyses were bootstrapped to account for error in estimating the weights. Results: ACT participants were fairly similar to older adults in the region. The largest differences were more self-reported current cholesterol medication use in BRFSS and higher proportions with low education in ACT. Incorporating the weights had little impact on prevalence estimates for age-related macular degeneration or glaucoma. Weighted estimates were slightly higher for diabetic retinopathy (weighted 5.7% (95% Confidence Interval 4.3, 7.1); unweighted 4.1% (3.6, 4.6)) and cataract history (weighted 51.8% (49.6, 54.3); unweighted 48.6% (47.3, 49.9)). The weighted hazard ratio for recent diabetic retinopathy diagnosis and Alzheimer's disease was 1.84 (0.34, 4.29), versus 1.32 (0.87, 2.00) in unweighted ACT. Conclusions: Most, but not all, associations were similar after participation weighting. Even in community-based cohorts, extending inferences to broader populations may benefit from evaluation with participation weights.

Person, Population, Mechanism. A Rejoinder to Critics and an Elaboration of the Three-Branch Model.

In a previous paper (Lundh, 2023), it was argued that psychological science can be seen as having three main branches, corresponding to three levels of research: research at the person level, at the population level, and at the mechanism level. The purpose of the present paper is to discuss the critique that has been raised against this model by Lamiell (2024) and Nilsson (2024) and to elaborate and specify the three-branch model in more detail. This is done by an incorporation of Nilsson's concept of person-sensitivity into the model, and by a clearer differentiation between the two contrasts involved: (1) the methodological focus either on individual persons or on populations of individuals; and (2) the theoretical focus either on whole-person functioning or on sub-personal mechanisms.

Why and how to collect representative study samples in educational neuroscience research.

BACKGROUND: Educational neuroscience research, which investigates the neurobiological mechanisms of learning, has historically incorporated samples drawn mostly from white, middle-class, and/or suburban populations. However, sampling in research without attending to representation can lead to biased interpretations and results that are less generalizable to an intended target population. Prior research revealing differences in neurocognitive outcomes both within- and across-groups further suggests that such practices may obscure significant effects with practical implications. BARRIERS: Negative attitudes among historically marginalized communities, stemming from historical mistreatment, biased research outcomes, and implicit or explicit attitudes among research teams, can hinder diverse participation. Qualities of the research process including language requirements, study locations, and time demands create additional barriers. SOLUTIONS: Flexible data collection approaches, community engaugement, and transparent reporting could build trust and enhance sampling diversity. Longer-term solutions include prioritizing research questions relevant to marginalized communities, increasing workforce diversity, and detailed reporting of sample demographics. Such concerted efforts are essential for robust educational neuroscience research to maximize positive impacts broadly across learners.

Improving the Generalizability and Performance of an Ultrasound Deep Learning Model Using Limited Multicenter Data for Lung Sliding Artifact Identification.

Deep learning (DL) models for medical image classification frequently struggle to generalize to data from outside institutions. Additional clinical data are also rarely collected to comprehensively assess and understand model performance amongst subgroups. Following the development of a single-center model to identify the lung sliding artifact on lung ultrasound (LUS), we pursued a validation strategy using external LUS data. As annotated LUS data are relatively scarce-compared to other medical imaging data-we adopted a novel technique to optimize the use of limited external data to improve model generalizability. Externally acquired LUS data from three tertiary care centers, totaling 641 clips from 238 patients, were used to assess the baseline generalizability of our lung sliding model. We then employed our novel Threshold-Aware Accumulative Fine-Tuning (TAAFT) method to fine-tune the baseline model and determine the minimum amount of data required to achieve predefined performance goals. A subgroup analysis was also performed and Grad-CAM++ explanations were examined. The final model was fine-tuned on one-third of the external dataset to achieve 0.917 sensitivity, 0.817 specificity, and 0.920 area under the receiver operator characteristic curve (AUC) on the external validation dataset, exceeding our predefined performance goals. Subgroup analyses identified LUS characteristics that most greatly challenged the model's performance. Grad-CAM++ saliency maps highlighted clinically relevant regions on M-mode images. We report a multicenter study that exploits limited available external data to improve the generalizability and performance of our lung sliding model while identifying poorly performing subgroups to inform future iterative improvements. This approach may contribute to efficiencies for DL researchers working with smaller quantities of external validation data.