Explainable nucleus classification using Decision Tree Approximation of Learned Embeddings.

MOTIVATION: Nucleus detection, segmentation and classification are fundamental to high-resolution mapping of the tumor microenvironment using whole-slide histopathology images. The growing interest in leveraging the power of deep learning to achieve state-of-the-art performance often comes at the cost of explainability, yet there is general consensus that explainability is critical for trustworthiness and widespread clinical adoption. Unfortunately, current explainability paradigms that rely on pixel saliency heatmaps or superpixel importance scores are not well-suited for nucleus classification. Techniques like Grad-CAM or LIME provide explanations that are indirect, qualitative and/or nonintuitive to pathologists. RESULTS: In this article, we present techniques to enable scalable nuclear detection, segmentation and explainable classification. First, we show how modifications to the widely used Mask R-CNN architecture, including decoupling the detection and classification tasks, improves accuracy and enables learning from hybrid annotation datasets like NuCLS, which contain mixtures of bounding boxes and segmentation boundaries. Second, we introduce an explainability method called Decision Tree Approximation of Learned Embeddings (DTALE), which provides explanations for classification model behavior globally, as well as for individual nuclear predictions. DTALE explanations are simple, quantitative, and can flexibly use any measurable morphological features that make sense to practicing pathologists, without sacrificing model accuracy. Together, these techniques present a step toward realizing the promise of computational pathology in computer-aided diagnosis and discovery of morphologic biomarkers. AVAILABILITY AND IMPLEMENTATION: Relevant code can be found at github.com/CancerDataScience/NuCLS. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Assessing clinical competence: a multitrait-multimethod matrix construct validity study.

Education in Doctor of Medicine programs has moved towards an emphasis on clinical competency, with entrustable professional activities providing a framework of learning objectives and outcomes to be assessed within the clinical environment. While the identification and structured definition of objectives and outcomes have evolved, many methods employed to assess clerkship students' clinical skills remain relatively unchanged. There is a paucity of medical education research applying advanced statistical design and analytic techniques to investigate the validity of clinical skills assessment. One robust statistical method, multitrait-multimethod matrix analysis, can be applied to investigate construct validity across multiple assessment instruments and settings. Four traits were operationalized to represent the construct of critical clinical skills (professionalism, data gathering, data synthesis, and data delivery). The traits were assessed using three methods (direct observations by faculty coaches, clinical workplace-based evaluations, and objective structured clinical examination type clinical practice examinations). The four traits and three methods were intercorrelated for the multitrait-multimethod matrix analysis. The results indicated reliability values in the adequate to good range across the three methods with the majority of the validity coefficients demonstrating statistical significance. The clearest evidence for convergent and divergent validity was with the professionalism trait. The correlations on the same method/different traits analyses indicated substantial method effect; particularly on clinical workplace-based assessments. The multitrait-multimethod matrix approach, currently underutilized in medical education, could be employed to explore validity evidence of complex constructs such as clinical skills. These results can inform faculty development programs to improve the reliability and validity of assessments within the clinical environment.

PodoSighter: A Cloud-Based Tool for Label-Free Podocyte Detection in Kidney Whole-Slide Images.

BACKGROUND: Podocyte depletion precedes progressive glomerular damage in several kidney diseases. However, the current standard of visual detection and quantification of podocyte nuclei from brightfield microscopy images is laborious and imprecise. METHODS: We have developed PodoSighter, an online cloud-based tool, to automatically identify and quantify podocyte nuclei from giga-pixel brightfield whole-slide images (WSIs) using deep learning. Ground-truth to train the tool used immunohistochemically or immunofluorescence-labeled images from a multi-institutional cohort of 122 histologic sections from mouse, rat, and human kidneys. To demonstrate the generalizability of our tool in investigating podocyte loss in clinically relevant samples, we tested it in rodent models of glomerular diseases, including diabetic kidney disease, crescentic GN, and dose-dependent direct podocyte toxicity and depletion, and in human biopsies from steroid-resistant nephrotic syndrome and from human autopsy tissues. RESULTS: The optimal model yielded high sensitivity/specificity of 0.80/0.80, 0.81/0.86, and 0.80/0.91, in mouse, rat, and human images, respectively, from periodic acid-Schiff-stained WSIs. Furthermore, the podocyte nuclear morphometrics extracted using PodoSighter were informative in identifying diseased glomeruli. We have made PodoSighter freely available to the general public as turnkey plugins in a cloud-based web application for end users. CONCLUSIONS: Our study demonstrates an automated computational approach to detect and quantify podocyte nuclei in standard histologically stained WSIs, facilitating podocyte research, and enabling possible future clinical applications.

Structured crowdsourcing enables convolutional segmentation of histology images.

MOTIVATION: While deep-learning algorithms have demonstrated outstanding performance in semantic image segmentation tasks, large annotation datasets are needed to create accurate models. Annotation of histology images is challenging due to the effort and experience required to carefully delineate tissue structures, and difficulties related to sharing and markup of whole-slide images. RESULTS: We recruited 25 participants, ranging in experience from senior pathologists to medical students, to delineate tissue regions in 151 breast cancer slides using the Digital Slide Archive. Inter-participant discordance was systematically evaluated, revealing low discordance for tumor and stroma, and higher discordance for more subjectively defined or rare tissue classes. Feedback provided by senior participants enabled the generation and curation of 20 000+ annotated tissue regions. Fully convolutional networks trained using these annotations were highly accurate (mean AUC=0.945), and the scale of annotation data provided notable improvements in image classification accuracy. AVAILABILITY AND IMPLEMENTATION: Dataset is freely available at: https://goo.gl/cNM4EL. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Determinants of medical specialty competitiveness.

BACKGROUND: Although the concept of medical specialty competitiveness may seem intuitive, there are very little existing empirical data on the determinants of specialty competitiveness in USA. An understanding of the determinants of specialty competitiveness may inform career choices among students and their advisors. Specialty competitiveness correlates with availability and appeal. METHODS: This narrative review examines 2019 National Resident Matching Program (NRMP) data and the existing literature to define the determinants of specialty competitiveness. A statistical analysis of key elements of the 2019 NRMP data was performed. RESULTS: Using US senior applicant fill rate as a measure of competitiveness, medical specialty competitiveness follows general principles of supply and demand. The demand, or appeal, of a specialty correlates with several factors, including salary, prestige and lifestyle. Salary correlates strongly with US senior fill rate (r=0.78, p=0.001). Relatively few positions are available for the most competitive specialties in the NRMP match. The negative correlation between US senior fill rate and position availability is also strong (r=-0.85; p<0.0001). CONCLUSION: A 'competitive specialty' correlates strongly with high earnings potential and limited position availability. In an ideal world, a student's pursuit of a medical specialty should be guided by interest, qualifications and ability to succeed in that field. However, students must contend with the realities of competition created by the residency matching system.

Validation of a performance checklist for ultrasound-guided internal jugular central lines for use in procedural instruction and assessment.

PURPOSE OF THE STUDY: Tools created to measure procedural competency must be tested in their intended environment against an established standard in order to be validated. We previously created a checklist for ultrasound-guided internal jugular central venous catheter (US IJ CVC) insertion using the modified Delphi method. We sought to further validate the checklist tool for use in an educational environment. STUDY DESIGN: This is a cohort study involving 15 emergency medicine interns being evaluated on their skill in US IJ CVC placement. We compared the checklist tool with a modified version of a clinically validated global rating scale (GRS) for procedural performance. RESULTS: The correlation between the GRS tool and the checklist tool was excellent, with a correlation coefficient (Pearson's r) of 0.90 (p<0.0001). CONCLUSIONS: This checklist represents a useful tool for measuring procedural competency.

Derivation of a Performance Checklist for Ultrasound-Guided Arthrocentesis Using the Modified Delphi Method.

OBJECTIVES: Arthrocentesis is an important skill for physicians in multiple specialties. Recent studies indicate a superior safety and performance profile for this procedure using ultrasound guidance for needle placement, and improving quality of care requires a valid measurement of competency using this modality. METHODS: We endeavored to create a validated tool to assess the performance of this procedure using the modified Delphi technique and experts in multiple disciplines across the United States. RESULTS: We derived a 22-item checklist designed to assess competency for the completion of ultrasound-guided arthrocentesis, which demonstrated a Cronbach's alpha of 0.89, indicating an excellent degree of internal consistency. CONCLUSIONS: Although we were able to demonstrate content validity for this tool, further validity evidence should be acquired after the tool is used and studied in clinical and simulated contexts.

Video-augmented feedback for procedural performance.

BACKGROUND: Resident programs must assess residents' achievement of core competencies for clinical and procedural skills. AIMS: Video-augmented feedback may facilitate procedural skill acquisition and promote more accurate self-assessment. METHODS: A randomized controlled study to investigate whether video-augmented verbal feedback leads to increased procedural skill and improved accuracy of self-assessment compared to verbal only feedback. Participants were evaluated during procedural training for ultrasound guided internal jugular central venous catheter (US IJ CVC) placement. All participants received feedback based on a validated 30-point checklist for US IJ CVC placement and validated 6-point procedural global rating scale. RESULTS: Scores in both groups improved by a mean of 9.6 points (95% CI: 7.8-11.4) on the 30-point checklist, with no difference between groups in mean score improvement on the global rating scale. In regards to self-assessment, participant self-rating diverged from faculty scoring, increasingly so after receiving feedback. Residents rated highly by faculty underestimated their skill, while those rated more poorly demonstrated increasing overestimation. CONCLUSIONS: Accuracy of self-assessment was not improved by addition of video. While feedback advanced the skill of the resident, video-augmented feedback did not enhance skill acquisition or improve accuracy of resident self-assessment compared to standard feedback.

Applying the master adaptive learner framework to just-in-time training of procedures.

Background: Just-in-time training (JITT) occurs in the clinical context when learners need immediate guidance for procedures due to a lack of proficiency or the need for knowledge refreshment. The master adaptive learner (MAL) framework presents a comprehensive model of transforming learners into adaptive experts, proficient not only in their current tasks but also in the ongoing development of lifelong skills. With the evolving landscape of procedural competence in emergency medicine (EM), trainees must develop the capacity to acquire and master new techniques consistently. This concept paper will discuss using JITT to support the development of MALs in the emergency department. Methods: In May 2023, an expert panel from the Society for Academic Emergency Medicine (SAEM) Medical Educator's Boot Camp delivered a comprehensive half-day preconference session entitled "Be the Best Teacher" at the society's annual meeting. A subgroup within this panel focused on applying the MAL framework to JITT. This subgroup collaboratively developed a practical guide that underwent iterative review and refinement. Results: The MAL-JITT framework integrates the learner's past experiences with the educator's proficiency, allowing the educational experience to address the unique requirements of each case. We outline a structured five-step process for applying JITT, utilizing the lumbar puncture procedure as an example of integrating the MAL stages of planning, learning, assessing, and adjusting. This innovative approach facilitates prompt procedural competence and cultivates a positive learning environment that fosters acquiring adaptable learning skills with enduring benefits throughout the learner's career trajectory. Conclusions: JITT for procedures holds the potential to cultivate a dynamic learning environment conducive to nurturing the development of MALs in EM.

Correction: Clinical reasoning education in the clerkship years: A cross-disciplinary national needs assessment.

[This corrects the article DOI: 10.1371/journal.pone.0273250.].

IFEM model curriculum: emergency medicine learning outcomes for undergraduate medical education.

BACKGROUND: The International Federation for Emergency Medicine (IFEM) published its model curriculum for medical student education in emergency medicine in 2009. Because of the evolving principles of emergency medicine and medical education, driven by societal, professional, and educational developments, there was a need for an update on IFEM recommendations. The main objective of the update process was creating Intended Learning Outcomes (ILOs) and providing tier-based recommendations. METHOD: A consensus methodology combining nominal group and modified Delphi methods was used. The nominal group had 15 members representing eight countries in six regions. The process began with a review of the 2009 curriculum by IFEM Core Curriculum and Education Committee (CCEC) members, followed by a three-phase update process involving survey creation [The final survey document included 55 items in 4 sections, namely, participant & context information (16 items), intended learning outcomes (6 items), principles unique to emergency medicine (20 items), and content unique to emergency medicine (13 items)], participant selection from IFEM member countries and survey implementation, and data analysis to create the recommendations. RESULTS: Out of 112 invitees (CCEC members and IFEM member country nominees), 57 (50.9%) participants from 27 countries participated. Eighteen (31.6%) participants were from LMICs, while 39 (68.4%) were from HICs. Forty-four (77.2%) participants have been involved with medical students' emergency medicine training for more than five years in their careers, and 56 (98.2%) have been involved with medical students' training in the last five years. Thirty-five (61.4%) participants have completed a form of training in medical education. The exercise resulted in the formulation of tiered ILO recommendations. Tier 1 ILOs are recommended for all medical schools, Tier 2 ILOs are recommended for medical schools based on perceived local healthcare system needs and/or adequate resources, and Tier 3 ILOs should be considered for medical schools based on perceived local healthcare system needs and/or adequate resources. CONCLUSION: The updated IFEM ILO recommendations are designed to be applicable across diverse educational and healthcare settings. These recommendations aim to provide a clear framework for medical schools to prepare graduates with essential emergency care capabilities immediately after completing medical school. The successful distribution and implementation of these recommendations hinge on support from faculty and administrators, ensuring that future healthcare professionals are well-prepared for emergency medical care.

FUSION: A web-based application for in-depth exploration of multi-omics data with brightfield histology.

Spatial -OMICS technologies facilitate the interrogation of molecular profiles in the context of the underlying histopathology and tissue microenvironment. Paired analysis of histopathology and molecular data can provide pathologists with otherwise unobtainable insights into biological mechanisms. To connect the disparate molecular and histopathologic features into a single workspace, we developed FUSION (Functional Unit State IdentificatiON in WSIs [Whole Slide Images]), a web-based tool that provides users with a broad array of visualization and analytical tools including deep learning-based algorithms for in-depth interrogation of spatial -OMICS datasets and their associated high-resolution histology images. FUSION enables end-to-end analysis of functional tissue units (FTUs), automatically aggregating underlying molecular data to provide a histopathology-based medium for analyzing healthy and altered cell states and driving new discoveries using "pathomic" features. We demonstrate FUSION using 10x Visium spatial transcriptomics (ST) data from both formalin-fixed paraffin embedded (FFPE) and frozen prepared datasets consisting of healthy and diseased tissue. Through several use-cases, we demonstrate how users can identify spatial linkages between quantitative pathomics, qualitative image characteristics, and spatial --omics.

Taking More Society for Academic Emergency Medicine Practice Tests Does Not Lead to Improved National EM-M4 Exam Scores.

INTRODUCTION: Emergency medicine (EM) is a required clerkship for third-year medical students, and an elective EM acting internship (AI) is available to fourth-year students at our institution. The Society for Academic Emergency Medicine's (SAEM) National Emergency Medicine M4 Examination (EM-M4) is administered to students at the end of the EM AI experience. To prepare for the exam, students gain access to 23 practice tests available from SAEM. In this study we investigate the correlation between the number of practice tests taken and EM-M4 performance. METHODS: We collected data for EM-M4 and the US Medical Licensing Exam (USMLE) Step 2 Clinical Knowledge (CK) from students completing a MS4 EM clerkship in consecutive medical school classes from 2014-2017 at a private medical school. In addition, we collected data during the clerkship on the number of practice exams taken and whether a comprehensive practice exam was taken. We analyzed the study population three ways to determine whether the number of practice tests impacted final exam results: a binary distribution (1-11 or 12-23 tests taken); quaternary distribution (1-6, 7-12, 13-18, or 19-23 tests taken); and individual test variability (1,2,3,…22,23 tests taken). Complete data for 147 students was used for data analysis. RESULTS: The EM-M4 showed moderate (r = 0.49) correlations with USMLE Step 2 CK. There was no significant difference in EM-M4 performance in the binary analysis (P ≤ 0.09), the quaternary analysis (P ≤ 0.09), or the continuous variable analysis (P ≤ 0.52). Inclusion of a comprehensive practice test also did not correlate with EM-M4 performance (P ≤ 0.78). CONCLUSION: Degree of utilization of SAEM practice tests did not seem to correlate with performance on the EM-M4 examination at our institution. This could be due to many factors including that the question bank is composed of items that had poor item discrimination, possible inadequate coverage of EM curriculum, and/or use of alternative study methods. While further investigation is needed, if our conclusions prove generalizable, then using the SAEM practice tests is an extraneous cognitive load from a modality without proven benefit.

Development of a lecture evaluation tool rooted in cognitive load theory: A modified Delphi study.

Background: Didactics play a key role in medical education. There is no standardized didactic evaluation tool to assess quality and provide feedback to instructors. Cognitive load theory provides a framework for lecture evaluations. We sought to develop an evaluation tool, rooted in cognitive load theory, to assess quality of didactic lectures. Methods: We used a modified Delphi method to achieve expert consensus for items in a lecture evaluation tool. Nine emergency medicine educators with expertise in cognitive load participated in three modified Delphi rounds. In the first two rounds, experts rated the importance of including each item in the evaluation rubric on a 1 to 9 Likert scale with 1 labeled as "not at all important" and 9 labeled as "extremely important." In the third round, experts were asked to make a binary choice of whether the item should be included in the final evaluation tool. In each round, the experts were invited to provide written comments, edits, and suggested additional items. Modifications were made between rounds based on item scores and expert feedback. We calculated descriptive statistics for item scores. Results: We completed three Delphi rounds, each with 100% response rate. After Round 1, we removed one item, made major changes to two items, made minor wording changes to nine items, and modified the scale of one item. Following Round 2, we eliminated three items, made major wording changes to one item, and made minor wording changes to one item. After the third round, we made minor wording changes to two items. We also reordered and categorized items for ease of use. The final evaluation tool consisted of nine items. Conclusions: We developed a lecture assessment tool rooted in cognitive load theory specific to medical education. This tool can be applied to assess quality of instruction and provide important feedback to speakers.

Validity evidence for an instrument for cognitive load for virtual didactic sessions.

BACKGROUND: COVID necessitated the shift to virtual resident instruction. The challenge of learning via virtual modalities has the potential to increase cognitive load. It is important for educators to reduce cognitive load to optimize learning, yet there are few available tools to measure cognitive load. The objective of this study is to identify and provide validity evidence following Messicks' framework for an instrument to evaluate cognitive load in virtual emergency medicine didactic sessions. METHODS: This study followed Messicks' framework for validity including content, response process, internal structure, and relationship to other variables. Content validity evidence included: (1) engagement of reference librarian and literature review of existing instruments; (2) engagement of experts in cognitive load, and relevant stakeholders to review the literature and choose an instrument appropriate to measure cognitive load in EM didactic presentations. Response process validity was gathered using the format and anchors of instruments with previous validity evidence and piloting amongst the author group. A lecture was provided by one faculty to four residency programs via ZoomTM. Afterwards, residents completed the cognitive load instrument. Descriptive statistics were collected; Cronbach's alpha assessed internal consistency of the instrument; and correlation for relationship to other variables (quality of lecture). RESULTS: The 10-item Leppink Cognitive Load instrument was selected with attention to content and response process validity evidence. Internal structure of the instrument was good (Cronbach's alpha = 0.80). Subscales performed well-intrinsic load (α = 0.96, excellent), extrinsic load (α = 0.89, good), and germane load (α = 0.97, excellent). Five of the items were correlated with overall quality of lecture (p < 0.05). CONCLUSIONS: The 10-item Cognitive Load instrument demonstrated good validity evidence to measure cognitive load and the subdomains of intrinsic, extraneous, and germane load. This instrument can be used to provide feedback to presenters to improve the cognitive load of their presentations.

A cloud-based tool for federated segmentation of whole slide images.

It is commonly known that diverse datasets of WSIs are beneficial when training convolutional neural networks, however sharing medical data between institutions is often hindered by regulatory concerns. We have developed a cloud-based tool for federated WSI segmentation, allowing collaboration between institutions without the need to directly share data. To show the feasibility of federated learning on pathology data in the real world, We demonstrate this tool by segmenting IFTA from three institutions and show that keeping the three datasets separate does not hinder segmentation performance. This pipeline is deployed in the cloud for easy access for data viewing and annotation by each site's respective constituents.

Clinical reasoning education in the clerkship years: A cross-disciplinary national needs assessment.

BACKGROUND: Improving clinical reasoning education has been identified as an important strategy to reduce diagnostic error-an important cause of adverse patient outcomes. Clinical reasoning is fundamental to each specialty, yet the extent to which explicit instruction in clinical reasoning occurs across specialties in the clerkship years remains unclear. METHOD: The Alliance for Clinical Education (ACE) Clinical Reasoning Workgroup and the Directors of Clinical Skills Courses (DOCS) Clinical Reasoning Workgroup collaborated to develop a clinical reasoning needs assessment survey. The survey questionnaire covered seven common clinical reasoning topics including illness scripts, semantic qualifiers, cognitive biases and dual process theory. Questionnaires were delivered electronically through ACE member organizations, which are primarily composed of clerkship leaders across multiple specialties. Data was collected between March of 2019 and May of 2020. RESULTS: Questionnaires were completed by 305 respondents across the six organizations. For each of the seven clinical reasoning topics, the majority of clerkship leaders (range 77.4% to 96.8%) rated them as either moderately important or extremely important to cover during the clerkship curriculum. Despite this perceived importance, these topics were not consistently covered in respondents' clerkships (range 29.4% to 76.4%) and sometimes not covered anywhere in the clinical curriculum (range 5.1% to 22.9%). CONCLUSIONS: Clerkship educators across a range of clinical specialties view clinical reasoning instruction as important, however little curricular time is allocated to formally teach the various strategies. Faculty development and restructuring of curricular time may help address this potential gap.

A tool for federated training of segmentation models on whole slide images.

The largest bottleneck to the development of convolutional neural network (CNN) models in the computational pathology domain is the collection and curation of diverse training datasets. Training CNNs requires large cohorts of image data, and model generalizability is dependent on training data heterogeneity. Including data from multiple centers enhances the generalizability of CNN-based models, but this is hindered by the logistical challenges of sharing medical data. In this paper, we explore the feasibility of training our recently developed cloud-based segmentation tool (Histo-Cloud) using federated learning. Using a dataset of renal tissue biopsies we show that federated training to segment interstitial fibrosis and tubular atrophy (IFTA) using datasets from three institutions is not found to be different from a training by pooling the data on one server when tested on a fourth (holdout) institution's data. Further, training a model to segment glomeruli for a federated dataset (split by staining) demonstrates similar performance.

PodoCount: A Robust, Fully Automated, Whole-Slide Podocyte Quantification Tool.

Introduction: Podocyte depletion is a histomorphologic indicator of glomerular injury and predicts clinical outcomes. Podocyte estimation methods or podometrics are semiquantitative, technically involved, and laborious. Implementation of high-throughput podometrics in experimental and clinical workflows necessitates an automated podometrics pipeline. Recognizing that computational image analysis offers a robust approach to study cell and tissue structure, we developed and validated PodoCount (a computational tool for automated podocyte quantification in immunohistochemically labeled tissues) using a diverse data set. Methods: Whole-slide images (WSIs) of tissues immunostained with a podocyte nuclear marker and periodic acid-Schiff counterstain were acquired. The data set consisted of murine whole kidney sections (n = 135) from 6 disease models and human kidney biopsy specimens from patients with diabetic nephropathy (DN) (n = 45). Within segmented glomeruli, podocytes were extracted and image analysis was applied to compute measures of podocyte depletion and nuclear morphometry. Computational performance evaluation and statistical testing were performed to validate podometric and associated image features. PodoCount was disbursed as an open-source, cloud-based computational tool. Results: PodoCount produced highly accurate podocyte quantification when benchmarked against existing methods. Podocyte nuclear profiles were identified with 0.98 accuracy and segmented with 0.85 sensitivity and 0.99 specificity. Errors in podocyte count were bounded by 1 podocyte per glomerulus. Podocyte-specific image features were found to be significant predictors of disease state, proteinuria, and clinical outcome. Conclusion: PodoCount offers high-performance podocyte quantitation in diverse murine disease models and in human kidney biopsy specimens. Resultant features offer significant correlation with associated metadata and outcome. Our cloud-based tool will provide end users with a standardized approach for automated podometrics from gigapixel-sized WSIs.