Recommendations for Performance Evaluation of Machine Learning in Pathology: A Concept Paper From the College of American Pathologists.

CONTEXT.­: Machine learning applications in the pathology clinical domain are emerging rapidly. As decision support systems continue to mature, laboratories will increasingly need guidance to evaluate their performance in clinical practice. Currently there are no formal guidelines to assist pathology laboratories in verification and/or validation of such systems. These recommendations are being proposed for the evaluation of machine learning systems in the clinical practice of pathology. OBJECTIVE.­: To propose recommendations for performance evaluation of in vitro diagnostic tests on patient samples that incorporate machine learning as part of the preanalytical, analytical, or postanalytical phases of the laboratory workflow. Topics described include considerations for machine learning model evaluation including risk assessment, predeployment requirements, data sourcing and curation, verification and validation, change control management, human-computer interaction, practitioner training, and competency evaluation. DATA SOURCES.­: An expert panel performed a review of the literature, Clinical and Laboratory Standards Institute guidance, and laboratory and government regulatory frameworks. CONCLUSIONS.­: Review of the literature and existing documents enabled the development of proposed recommendations. This white paper pertains to performance evaluation of machine learning systems intended to be implemented for clinical patient testing. Further studies with real-world clinical data are encouraged to support these proposed recommendations. Performance evaluation of machine learning models is critical to verification and/or validation of in vitro diagnostic tests using machine learning intended for clinical practice.

Entrustable Professional Activities for Pathology: Recommendations From the College of American Pathologists Graduate Medical Education Committee.

Competency-based medical education has evolved over the past decades to include the Accreditation Council for Graduate Medical Education Accreditation System of resident evaluation based on the Milestones project. Entrustable professional activities represent another means to determine learner proficiency and evaluate educational outcomes in the workplace and training environment. The objective of this project was to develop entrustable professional activities for pathology graduate medical education encompassing primary anatomic and clinical pathology residency training. The Graduate Medical Education Committee of the College of American Pathologists met over the course of 2 years to identify and define entrustable professional activities for pathology graduate medical education. Nineteen entrustable professional activities were developed, including 7 for anatomic pathology, 4 for clinical pathology, and 8 that apply to both disciplines with 5 of these concerning laboratory management. The content defined for each entrustable professional activity includes the entrustable professional activity title, a description of the knowledge and skills required for competent performance, mapping to relevant Accreditation Council for Graduate Medical Education Milestone subcompetencies, and general assessment methods. Many critical activities that define the practice of pathology fit well within the entrustable professional activity model. The entrustable professional activities outlined by the Graduate Medical Education Committee are meant to provide an initial framework for the development of entrustable professional activity-related assessment and curricular tools for pathology residency training.

Perceptions of Unprofessional Attitudes and Behaviors: Implications for Faculty Role Modeling and Teaching Professionalism During Pathology Residency.

CONTEXT: - Changes occurring in medicine have raised issues about medical professionalism. Professionalism is included in the Core Competencies and Milestones for all pathology residents. Previous studies have looked at resident professionalism attitudes and behaviors in primary care but none have looked specifically at pathology. OBJECTIVE: - To examine behavior and attitudes toward professionalism within pathology and to determine how professionalism is taught in residency programs. DESIGN: - Surveys were sent to all College of American Pathologists junior members and all pathology residency program directors, and responses were compared. RESULTS: - Although no single behavior received the same professionalism rating among residents and program directors, both groups identified the same behaviors as being the most unprofessional: posting identifiable patient information or case images to social media, making a disparaging comment about a physician colleague or member of the support staff on social media or in a public hospital space, and missing work without reporting the time off. Faculty were observed displaying most of these behaviors as often or more often than residents by both groups. The most common means to teach professionalism in pathology residencies is providing feedback as situations arise and teaching by example. Age differences were found within each group and between groups for observed behaviors and attitudes. CONCLUSIONS: - As teaching by example was identified as a common educational method, faculty must be aware of the role their behavior and attitudes have in shaping resident behavior and attitudes. These results suggest a need for additional resources to teach professionalism during pathology residency.

Employer Expectations for Newly Trained Pathologists: Report of a Survey From the Graduate Medical Education Committee of the College of American Pathologists.

CONTEXT: -Multiple sources have identified challenges that training programs face in preparing graduates for the "real world" practice of pathology, and many training programs have sought to decrease the gap between skills acquired during training and those required in practice. However, there exists the possibility that some of the difficulty experienced by newly trained pathologists and employers might arise from differences between employer expectations of new hires and what applicants expect from their first job. OBJECTIVE: -To define the constellation of skills and attributes employers prioritize when hiring newly trained pathologists. DESIGN: -A survey of fellows of the College of American Pathologists in practice for 5 or more years in the United States was administered and the results were analyzed. RESULTS: -A total of 630 pathologists who were responsible for hiring a new-in-practice pathologist completed the survey. Regardless of practice setting, certain skills and attributes were rated critically important in new hires, including ethics/integrity, work ethic, and professionalism. Seventy-one percent reported having some difficulty hiring entry-level pathologists and cited inadequate training/experience during residency, and applicants having unrealistic expectations regarding work load/hours as the most common reasons. CONCLUSIONS: -Prospective employers not only expect well-developed diagnostic skills in their job applicants, but also require evidence of a strong work ethic and outstanding professionalism. Successful applicants must display willingness to assume responsibilities and flexibility regarding existing and new responsibilities. A secondary but important finding of this survey was that most jobs are garnered through word-of-mouth recommendations; therefore, it is crucial for pathologists-in-training to hone their networking skills.

Professionalism in Pathology: A Case-Based Approach as a Potential Educational Tool.

CONTEXT: -Professionalism issues in residency training can be difficult to assess and manage. Generational or role-based differences may also exist between faculty and residents as to what constitutes unprofessional behavior and how to manage it. OBJECTIVE: -To examine and compare how faculty and residents would approach the same 5 case scenarios detailing various aspects of unprofessional behavior. DESIGN: -Five case scenarios highlighting various unprofessional behaviors were presented in a workshop at an annual meeting of pathology department chairs, residency program directors, and undergraduate pathology medical educators (ie, pathologists involved in medical student pathology education). The same cases were presented to a cohort of pathology residents currently in training. A standard set of responses were offered to the participants, polling results were collected electronically, and results were compared. RESULTS: -Faculty and residents were fairly consistent within their respective groups. In a subset of cases, faculty were more likely to favor working with the individual in the scenario, whereas resident respondents were more likely to favor either no response or a severe response. Generational or role-based differences were also potentially evident. CONCLUSIONS: -Assessing expectations and differences around professionalism for both faculty and residents should be considered as part of any educational and management approach for professionalism. Although a level of generational differences appears to be evident in this study regarding the recognition and management of unprofessional behavior, there was also agreement in some cases. Further exploration into the discrepant responses between faculty and residents may prove useful in developing educational, assessment, and remediation resources.

Assessment and Management of Professionalism Issues in Pathology Residency Training: Results From Surveys and a Workshop by the Graduate Medical Education Committee of the College of American Pathologists.

Professionalism issues are common in residency training and can be very difficult to recognize and manage. Almost one-third of the milestones for pathology recently instituted by the Accreditation Council for Graduate Medical Education encompass aspects of professionalism. Program directors are often unsure of how and when to remediate residents for unprofessional behavior. We used a case-based educational approach in a workshop setting to assist program directors in the management of unprofessional behavior in residents. Eight case scenarios highlighting various aspects of unprofessional behavior by pathology residents were developed and presented in an open workshop forum at the annual pathology program director's meeting. Prior to the workshop, 2 surveys were conducted: (1) to collect data on program directors' experience with identifying, assessing, and managing unprofessional behavior in their residents and (2) to get feedback from workshop registrants on how they would manage each of the 8 case scenarios. A wide range of unprofessional behaviors have been observed by pathology program directors. Although there is occasionally general agreement on how to manage specific behaviors, there remains wide variation in how to manage many of the presented unprofessional behaviors. Remediation for unprofessional behavior in pathology residents remains a difficult and challenging process. Additional education and research in this area are warranted.

Acquired hemoglobin variants and exposure to glucose-6-phosphate dehydrogenase deficient red blood cell units during exchange transfusion for sickle cell disease in a patient requiring antigen-matched blood.

Red blood cell exchange (RBCEx) is frequently used in the management of patients with sickle cell disease (SCD) and acute chest syndrome or stroke, or to maintain target hemoglobin S (HbS) levels. In these settings, RBCEx is a category I or II recommendation according to guidelines on the use of therapeutic apheresis published by the American Society for Apheresis. Matching donor red blood cells (RBCs) to recipient phenotypes (e.g., C, E, K-antigen negative) can decrease the risk of alloimmunization in patients with multi-transfused SCD. However, this may select for donors with a higher prevalence of RBC disorders for which screening is not performed. This report describes a patient with SCD treated with RBCEx using five units negative for C, E, K, Fya, Fyb (prospectively matched), four of which were from donors with hemoglobin variants and/or glucose-6-phosphate dehydrogenase (G6PD) deficiency. Pre-RBCEx HbS quantification by high performance liquid chromatography (HPLC) demonstrated 49.3% HbS and 2.8% hemoglobin C, presumably from transfusion of a hemoglobin C-containing RBC unit during a previous RBCEx. Post-RBCEx HPLC showed the appearance of hemoglobin G-Philadelphia. Two units were G6PD-deficient. The patient did well, but the consequences of transfusing RBC units that are G6PD-deficient and contain hemoglobin variants are unknown. Additional studies are needed to investigate effects on storage, in-vivo RBC recovery and survival, and physiological effects following transfusion of these units. Post-RBCEx HPLC can monitor RBCEx efficiency and detect the presence of abnormal transfused units.

Short-term fasting-induced autonomic activation and changes in catecholamine levels are not mediated by changes in leptin levels in healthy humans.

OBJECTIVE: In animal models, the adipocyte-secreted hormone leptin increases energy expenditure by increasing sympathetic outflow but its role in humans remains to be elucidated. We evaluated whether inducing hypoleptinaemia (with and without administration of leptin at replacement doses) for 3 days would influence catecholamine levels and sympathetic and parasympathetic activity in healthy humans. METHODS: We studied six normal-weight subjects in the General Clinical Research Center (GCRC) under three conditions: baseline fed state (control study) and two 72-h fasting studies (to decrease leptin levels), with administration of either placebo or replacement-dose recombinant methionyl human leptin (r-metHuLeptin) in a randomized, double-blind fashion. In each condition, 24-h urinary catecholamine levels, heart rate and heart rate variability (HRV), a standard tool for assessing cardiac autonomic modulation, were measured. RESULTS: Study parameters remained stable during the control condition and the baseline assessment of all three studies. In response to 72-h fasting, which decreased serum leptin levels by 80%, 24-h urinary norepinephrine and dopamine levels and heart rate increased while cardiac vagal modulation decreased (all P < 0.05). Replacement-dose r-metHuLeptin to keep leptin levels within the physiological range during fasting did not alter fasting-associated changes in heart rate, catecholamine levels or cardiac vagal tone. CONCLUSIONS: The findings of this controlled, interventional study indicate that changes in heart rate, catecholamine levels and cardiac vagal modulation associated with 72-h fasting are independent of regulation by leptin. Thus, changes in leptin levels within the physiological range do not seem to play a role in regulating autonomic function during short-term starvation in healthy humans.