Identifying Community-Based Entrustable Professional Activities for Medical Students Through a Modified Delphi Process.

BACKGROUND AND OBJECTIVES: Recognition of the need for medical education to train physicians who are skilled at supporting population health and work beyond traditional health care settings is growing. Entrustable professional activities (EPAs) for medical students typically have centered around activities taking place in the clinical workplace; however, EPAs that involve working with community members in community contexts have not been clearly established. METHODS: We used a three-stage online modified-Delphi method to identify community-based EPAs for University of Wisconsin School of Medicine and Public Health medical students. We recruited key stakeholders to participate and asked them, based on their experience, to generate a list of community-based tasks that they believed graduates should be trusted to perform. Subsequently, using a five-point anchored Likert scale (1=strongly disagree to 5=strongly agree), we asked participants to rate their level of agreement with each identified task becoming an EPA. An a priori definition of consensus was established. RESULTS: Twenty-two tasks reached consensus as potential community-based EPAs. The tasks with the highest mean ratings were "addressing trust issues with the medical community amongst the local population" (mean=4.71), "meeting with community members around a health topic" (mean=4.64), "identifying opportunities for disease prevention" (mean=4.64), and "identifying policies that impact community outcomes" (mean=4.57). CONCLUSIONS: The identified community-based tasks can support the augmentation of existing community-based curriculum and help identify areas for novel curriculum and assessment development. Lessons learned from this local effort could be helpful to other programs seeking to establish and refine community-based curricula.

Targeted Training Reduces Search Errors but Not Classification Errors for Hepatic Metastasis Detection at Contrast-Enhanced CT.

RATIONALE AND OBJECTIVES: Methods are needed to improve the detection of hepatic metastases. Errors occur in both lesion detection (search) and decisions of benign versus malignant (classification). Our purpose was to evaluate a training program to reduce search errors and classification errors in the detection of hepatic metastases in contrast-enhanced abdominal computed tomography (CT). MATERIALS AND METHODS: After Institutional Review Board approval, we conducted a single-group prospective pretest-posttest study. Pretest and posttest were identical and consisted of interpreting 40 contrast-enhanced abdominal CT exams containing 91 liver metastases under eye tracking. Between pretest and posttest, readers completed search training with eye-tracker feedback and coaching to increase interpretation time, use liver windows, and use coronal reformations. They also completed classification training with part-task practice, rating lesions as benign or malignant. The primary outcome was metastases missed due to search errors (<2 seconds gaze under eye tracker) and classification errors (>2 seconds). Jackknife free-response receiver operator characteristic (JAFROC) analysis was also conducted. RESULTS: A total of 31 radiologist readers (8 abdominal subspecialists, 8 nonabdominal subspecialists, 15 senior residents/fellows) participated. Search errors were reduced (pretest 11%, posttest 8%, difference 3% [95% confidence interval, 0.3%-5.1%], P = .01), but there was no difference in classification errors (difference 0%, P = .97) or in JAFROC figure of merit (difference -0.01, P = .36). In subgroup analysis, abdominal subspecialists demonstrated no evidence of change. CONCLUSION: Targeted training reduced search errors but not classification errors for the detection of hepatic metastases at contrast-enhanced abdominal CT. Improvements were not seen in all subgroups.

A training program to reduce reader search errors for liver metastasis detection in CT.

Detection of low contrast liver metastases varies between radiologists. Training may improve performance for lower-performing readers and reduce inter-radiologist variability. We recruited 31 radiologists (15 trainees, 8 non-abdominal staff, and 8 abdominal staff) to participate in four separate reading sessions: pre-test, search training, classification training, and post-test. In the pre-test, each radiologist interpreted 40 liver CT exams containing 91 metastases, circumscribed suspected hepatic metastases while under eye tracker observation, and rated confidence. In search training, radiologists interpreted a separate set of 30 liver CT exams while receiving eye tracker feedback and after coaching to increase use of coronal reformations, interpretation time, and use of liver windows. In classification training, radiologists interpreted up to 100 liver CT image patches, most with benign or malignant lesions, and compared their annotations to ground truth. Post-test was identical to pre-test. Between pre- and post-test, sensitivity increased by 2.8% (p = 0.01) but AUC did not change significantly. Missed metastases were classified as search errors (<2 seconds gaze time) or classification errors (>2 seconds gaze time) using the eye tracker. Out of 2775 possible detections, search errors decreased (10.8% to 8.1%; p < 0.01) but classification errors were unchanged (5.7% vs 5.7%). When stratified by difficulty, easier metastases showed larger reductions in search errors: for metastases with average sensitivity of 0-50%, 50-90%, and 90-100%, reductions in search errors were 16%, 35%, and 58%, respectively. The training program studied here may be able to improve radiologist performance by reducing errors but not classification errors.

In Vivo Prediction of Kidney Stone Fragility Using Radiomics-Based Regression Models.

Introduction: The surgical technique for urinary stone removal is partly influenced by its fragility, as prognosticated by the clinician. This feasibility study aims to develop a linear regression model from CT-based radiomic markers to predict kidney stone comminution time in vivo with two ultrasonic lithotrites. Materials and Methods: Patients identified by urologists at our institution as eligible candidates for percutaneous nephrolithotomy were prospectively enrolled. The active engagement time of the lithotrite in breaking the stone during surgery denoted the comminution time of each stone. The comminution rate was computed as the stone volume disintegrated per minute. Stones were grouped into three fragility classes (fragile, moderate, hard), based on inverse of the comminution rates with respect to the mean. Multivariable linear regression models were trained with radiomic features extracted from clinical CT images to predict comminution times in vivo. The model with the least root mean squared error (RMSE) on comminution times and the fewest misclassification of fragility was finally selected. Results: Twenty-eight patients with 31 stones in total were included in this study. Stones in the cohort averaged 1557 (±2472) mm3 in volume and 5.3 (±7.4) minutes in comminution time. Ten stones had nonmoderate fragility. Linear regression of stone volume alone predicted comminution time with an RMSE of 6.8 minutes and missed all 10 stones with nonmoderate fragility. A fragility model that included stone volume, internal morphology, shape-based radiomics, and device type improved RMSE to below 3.3 minutes and correctly classified 20/21 moderate and 6/10 nonmoderate stones. Conclusions: CT metrics-based fragility models may provide information to surgeons regarding kidney stone fragility and facilitate the selection of stone removal procedures.

Understanding Reader Variability: A 25-Radiologist Study on Liver Metastasis Detection at CT.

Background Substantial interreader variability exists for common tasks in CT imaging, such as detection of hepatic metastases. This variability can undermine patient care by leading to misdiagnosis. Purpose To determine the impact of interreader variability associated with (a) reader experience, (b) image navigation patterns (eg, eye movements, workstation interactions), and (c) eye gaze time at missed liver metastases on contrast-enhanced abdominal CT images. Materials and Methods In a single-center prospective observational trial at an academic institution between December 2020 and February 2021, readers were recruited to examine 40 contrast-enhanced abdominal CT studies (eight normal, 32 containing 91 liver metastases). Readers circumscribed hepatic metastases and reported confidence. The workstation tracked image navigation and eye movements. Performance was quantified by using the area under the jackknife alternative free-response receiver operator characteristic (JAFROC-1) curve and per-metastasis sensitivity and was associated with reader experience and image navigation variables. Differences in area under JAFROC curve were assessed with the Kruskal-Wallis test followed by the Dunn test, and effects of image navigation were assessed by using the Wilcoxon signed-rank test. Results Twenty-five readers (median age, 38 years; IQR, 31-45 years; 19 men) were recruited and included nine subspecialized abdominal radiologists, five nonabdominal staff radiologists, and 11 senior residents or fellows. Reader experience explained differences in area under the JAFROC curve, with abdominal radiologists demonstrating greater area under the JAFROC curve (mean, 0.77; 95% CI: 0.75, 0.79) than trainees (mean, 0.71; 95% CI: 0.69, 0.73) (P = .02) or nonabdominal subspecialists (mean, 0.69; 95% CI: 0.60, 0.78) (P = .03). Sensitivity was similar within the reader experience groups (P = .96). Image navigation variables that were associated with higher sensitivity included longer interpretation time (P = .003) and greater use of coronal images (P < .001). The eye gaze time was at least 0.5 and 2.0 seconds for 71% (266 of 377) and 40% (149 of 377) of missed metastases, respectively. Conclusion Abdominal radiologists demonstrated better discrimination for the detection of liver metastases on abdominal contrast-enhanced CT images. Missed metastases frequently received at least a brief eye gaze. Higher sensitivity was associated with longer interpretation time and greater use of liver display windows and coronal images. © RSNA, 2022 Online supplemental material is available for this article.

Individualized and generalized models for predicting observer performance on liver metastasis detection using CT.

Purpose: Radiologists exhibit wide inter-reader variability in diagnostic performance. This work aimed to compare different feature sets to predict if a radiologist could detect a specific liver metastasis in contrast-enhanced computed tomography (CT) images and to evaluate possible improvements in individualizing models to specific radiologists. Approach: Abdominal CT images from 102 patients, including 124 liver metastases in 51 patients were reconstructed at five different kernels/doses using projection domain noise insertion to yield 510 image sets. Ten abdominal radiologists marked suspected metastases in all image sets. Potentially salient features predicting metastasis detection were identified in three ways: (i) logistic regression based on human annotations (semantic), (ii) random forests based on radiologic features (radiomic), and (iii) inductive derivation using convolutional neural networks (CNN). For all three approaches, generalized models were trained using metastases that were detected by at least two radiologists. Conversely, individualized models were trained using each radiologist's markings to predict reader-specific metastases detection. Results: In fivefold cross-validation, both individualized and generalized CNN models achieved higher area under the receiver operating characteristic curves (AUCs) than semantic and radiomic models in predicting reader-specific metastases detection ability ( p < 0.001 ). The individualized CNN with an AUC of mean (SD) 0.85(0.04) outperformed the generalized one [ AUC = 0.78 ( 0.06 ) , p = 0.004 ]. The individualized semantic [ AUC = 0.70 ( 0.05 ) ] and radiomic models [ AUC = 0.68 ( 0.06 ) ] outperformed the respective generalized versions [semantic AUC = 0.66 ( 0.03 ) , p = 0.009 ; radiomic AUC = 0.64 ( 0.06 ) , p = 0.03 ]. Conclusions: Individualized models slightly outperformed generalized models for all three feature sets. Inductive CNNs were better at predicting metastases detection than semantic or radiomic features. Generalized models have implementation advantages when individualized data are unavailable.

Individualized and Generalized Learner Models for Predicting Missed Hepatic Metastases.

The diagnostic performance of radiologist readers exhibits substantial variation that cannot be explained by CT acquisition protocol differences. Studying reader detectability from CT images may help identify why certain types of lesions are missed by multiple or specific readers. Ten subspecialized abdominal radiologists marked all suspected metastases in a multi-reader-multi-case study of 102 deidentified contrast-enhanced CT liver scans at multiple radiation dose levels. A reference reader marked ground truth metastatic and benign lesions with the aid of histopathology or tumor progression on later scans. Multi-slice image patches and 3D radiomic features were extracted from the CT images. We trained deep convolutional neural networks (CNN) to predict whether an average (generalized) or individual radiologist reader would detect or miss a specific metastasis from an image patch containing it. The individualized CNN showed higher performance with an area under the receiver operating characteristic curve (AUC) of 0.82 compared to a generalized one (AUC = 0.78) in predicting reader-specific detectability. Random forests were used to build the respective versions from radiomic features. Both the individualized (AUC = 0.64) and generalized (AUC = 0.59) predictors from radiomic features showed limited ability to differentiate detected from missed lesions. This shows that CNN can identify and learn automated features that are better predictors of reader detectability of lesions than radiomic features. Individualized prediction of difficult lesions may allow targeted training of idiosyncratic weaknesses but requires substantial training data for each reader.

A 25-reader performance study for hepatic metastasis detection: lessons from unsupervised learning.

There is substantial variability in the performance of radiologist readers. We hypothesized that certain readers may have idiosyncratic weaknesses towards certain types of lesions, and unsupervised learning techniques might identify these patterns. After IRB approval, 25 radiologist readers (9 abdominal subspecialists and 16 non-specialists or trainees) read 40 portal phase liver CT exams, marking all metastases and providing a confidence rating on a scale of 1 to 100. We formed a matrix of reader confidence ratings, with rows corresponding to readers, and columns corresponding to metastases, and each matrix entry providing the confidence rating that a reader gave to the metastasis, with zero confidence used for lesions that were not marked. A clustergram was used to permute the rows and columns of this matrix to group similar readers and metastases together. This clustergram was manually interpreted. We found a cluster of lesions with atypical presentation that were missed by several readers, including subspecialists, and a separate cluster of small, subtle lesions where subspecialists were more confident of their diagnosis than trainees. These and other observations from unsupervised learning could inform targeted training and education of future radiologists.

Modeling Multi-View Dependence in Bayesian Networks for Alzheimer's Disease Detection.

Early detection of Alzheimer's disease is important for deploying interventions to prevent or slow disease progression. We propose a multi-view dependence modeling framework that integrates multiple data sources to distinguish patients at different stages of the disease. We design interpretable models that can handle heterogeneous data types including neuro-images, bio- and clinical markers, and historical and genotypical characteristics of the subjects. We learn the dependence structure from data with guidance from domain knowledge in Bayesian Networks, visualizing and quantifying the conditional probabilistic dependence among the variables. Our results indicate that the hybrid dependence models also improve prediction performance.

Screening Pediatric Patients for Food Insecurity: A Retrospective Cross-Sectional Study of Comorbidities and Demographic Characteristics.

BACKGROUND: Food insecurity is a household-level economic and social condition of limited or uncertain access to adequate and nutritional food that is associated with diabetes, obesity, anxiety, depression, and behavioral disorders. The presence of these comorbidities motivated the UW Health Pediatrics Department to start screening for food insecurity. METHODS: Our study describes demographic characteristics of screened patients, comparing risk factors and health status between food insecure patients and food secure patients. We extracted variables on all screened patients: sex, age, race, ethnicity, insurance type, height, weight (to calculate body mass index [BMI] and BMI percentile), and any diagnosis of diabetes, hypertension, sleeping problems, restless leg syndrome, anemia, elevated blood lead levels, depression, anxiety, or attention deficit disorder/attention deficit hyperactivity disorder (ADD/ADHD). RESULTS: Over the 8-month screening period, 1,330 patients were screened for food insecurity, and 30 screened positive. Insurance type was a significant predictor for food insecurity; patients on public or with no insurance had 6.39 times greater odds of being food insecure than those on private insurance (CI 3.81, 13.29). Also, diagnoses of anemia and ADD/ADHD were both significantly higher in the food insecure group. The odds of having anemia was 8.47 times greater for food insecure patients (CI 3.03, 23.63), and the odds for having ADD/ADHD was 5.89 times greater for food insecure patients than food secure patients (CI 1.48, 23.55). DISCUSSION: These results provide useful information to clinicians as the screening process moves toward widespread adoption. These results also provide a baseline for expanded research once screening is implemented throughout all pediatric clinics within our health care organization.

Knowledge-Driven Interpretation of Multi-View Data in Medicine.

We propose a new approach to clinical decision support with interpretable recommendations from multi-view data. We introduce a Bayesian network structure learning method to help identify the relevant factors and their relationships. Guided by minimal domain knowledge, this method highlights the significant associations among the demography, medical and family history, lifestyle, and biomarker data to facilitate informed clinical decisions. We demonstrate the effectiveness of the method for detecting mild neurocognitive disorder in the elderly from a real-life dataset in Singapore. The empirical results show that our method achieves better interpretability in addition to comparable accuracy with respect to the benchmark studies.

Inflammatory Papillitis in Uveitis: Response to Treatment and Use of Optic Nerve Optical Coherence Tomography for Monitoring.

PURPOSE: To describe the clinical course of uveitis-associated inflammatory papillitis and evaluate the utility and reproducibility of optic nerve spectral domain optical coherence tomography (SD-OCT). METHODS: Data on 22 eyes of 14 patients with uveitis-related papillitis and optic nerve imaging were reviewed. SD-OCT measure reproducibility was determined and parameters were compared in active vs. inactive uveitis. RESULTS: Papillitis resolution lagged behind uveitis resolution in three patients. For SD-OCT measures, the intraclass correlation coefficients were 99.1-100% and 86.9-100% for intraobserver and interobserver reproducibility, respectively. All SD-OCT optic nerve measures except inferior and nasal peripapillary retinal thicknesses were significantly higher in active vs. inactive uveitis after correction for multiple hypotheses testing. Mean optic nerve central thickness decreased from 545.1 to 362.9 µm (p = 0.01). CONCLUSIONS: Resolution of inflammatory papillitis can lag behind resolution of uveitis. SD-OCT assessment of papillitis is reproducible and correlates with presence vs. resolution of uveitis.

Development of an Obesity Prevention Dashboard for Wisconsin.

IMPORTANCE: A comprehensive obesity surveillance system monitors obesity rates along with causes and related health policies, which are valuable for tracking and identifying problems needing intervention. METHODS: A statewide obesity dashboard was created using the County Health Rankings model. Indicators were obtained through publicly available secondary data sources and used to rank Wisconsin amongst other states on obesity rates, health factors, and policies. RESULTS: Wisconsin consistently ranks in the middle of states for a majority of indicators and has not implemented any of the evidence-based health policies. CONCLUSIONS AND RELEVANCE: This state of obesity report shows Wisconsin has marked room for improvement regarding obesity prevention, especially with obesity-related health policies. Physicians and health care systems can play a pivotal role in making progress on obesity prevention.

Prevalence and Predictors of Unhealthy Weight Gain in Pregnancy.

IMPORTANCE: Weight gain during pregnancy affects obesity risk in offspring. OBJECTIVE: To assess weight gain among UW Health prenatal patients and to identify predictors of unhealthy gestational weight gain. METHODS: Retrospective cohort study of women delivering at UW Health during 2007-2012. Data are from the UW eHealth Public Health Information Exchange (PHINEX) project. The proportion of women with excess and insufficient (ie, unhealthy) gestational weight gain was computed based on 2009 Institute of Medicine guidelines. Multivariable logistic regression was used to identify risk factors associated with excess and insufficient gestational weight gain. RESULTS: Gestational weight gain of 7,385 women was analyzed. Fewer than 30% of prenatal patients gained weight in accordance with Institute of Medicine guidelines. Over 50% of women gained excess weight and 20% gained insufficient weight during pregnancy. Pre-pregnancy weight and smoking status predicted excess weight gain. Maternal age, race/ethnicity, smoking status, and having Medicaid insurance predicted insufficient weight gain. CONCLUSIONS AND RELEVANCE: Unhealthy weight gain during pregnancy is the norm for Wisconsin women. Clinical and community interventions that promote healthy weight gain during pregnancy will not only improve the health of mothers, but also will reduce the risk of obesity in the next generation.

Fusing Heterogeneous Data for Alzheimer's Disease Classification.

In multi-view learning, multimodal representations of a real world object or situation are integrated to learn its overall picture. Feature sets from distinct data sources carry different, yet complementary, information which, if analysed together, usually yield better insights and more accurate results. Neuro-degenerative disorders such as dementia are characterized by changes in multiple biomarkers. This work combines the features from neuroimaging and cerebrospinal fluid studies to distinguish Alzheimer's disease patients from healthy subjects. We apply statistical data fusion techniques on 101 subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. We examine whether fusion of biomarkers helps to improve diagnostic accuracy and how the methods compare against each other for this problem. Our results indicate that multimodal data fusion improves classification accuracy.

Beyond content: leadership development through a journal club.

CDC designed its Health Systems Integration Program to prepare leaders to function at the interface of public health and health care. Specific Health Systems Integration Program competencies in the areas of communication, analysis and assessment, and health systems were developed to nurture evidence-based decision-making and leadership skills crucial for future public health leaders. The program therefore designed an innovative journal club as part of its competency-based curriculum not only to meet the standard goals for a journal club-critical reading, interpretation, and acquiring content knowledge-but also to foster leadership development. This report describes the Health Systems Integration Program journal club format, its implementation, challenges, and key elements of success. Other programs using a journal club model as a learning format might consider using the Health Systems Integration Program's innovative approach that focuses on leadership development.

Blau syndrome-associated uveitis and the NOD2 gene.

Blau syndrome (BS), a rare autosomal dominant autoinflammatory syndrome, is an example of a monogenic disease. It was first described as a classic triad of uveitis, arthritis, and exanthema, typically seen in patients less than four years of age. Since that time, the phenotype has been expanded to include fever, cranial neuropathies, cardiovascular abnormalities, and granulomas of the liver and kidney. The ocular inflammation is often a panuveitis that occurs later in the disease course and typically carries the greatest morbidity in BS. BS has been mapped to the chromosomal region 16q12-21, also known as the NOD2 gene (formerly CARD15/NOD2). The disease is secondary to a single amino acid mutation NOD2 that leads to peptidoglycan-independent activity of nuclear factor (NF)-κB. Clinical and genetic aspects of BS will be discussed, as well as recent advances in treatment protocols.