Learner Assessment and Program Evaluation: Supporting Precision Education.

ABSTRACT: Precision education (PE) systematically leverages data and advanced analytics to inform educational interventions that, in turn, promote meaningful learner outcomes. PE does this by incorporating analytic results back into the education continuum through continuous feedback cycles. These data-informed sequences of planning, learning, assessing, and adjusting foster competence and adaptive expertise. PE cycles occur at individual (micro), program (meso), or system (macro) levels. This article focuses on program- and system-level PE.Data for PE come from a multitude of sources, including learner assessment and program evaluation. The authors describe the link between these data and the vital role evaluation plays in providing evidence of educational effectiveness. By including prior program evaluation research supporting this claim, the authors illustrate the link between training programs and patient outcomes. They also describe existing national reports providing feedback to programs and institutions, as well as 2 emerging, multiorganization program- and system-level PE efforts. The challenges encountered by those implementing PE and the continuing need to advance this work illuminate the necessity for increased cross-disciplinary collaborations and a national cross-organizational data-sharing effort.Finally, the authors propose practical approaches for funding a national initiative in PE as well as potential models for advancing the field of PE. Lessons learned from successes by others illustrate the promise of these recommendations.

Artificial Intelligence Screening of Medical School Applications: Development and Validation of a Machine-Learning Algorithm.

PURPOSE: To explore whether a machine-learning algorithm could accurately perform the initial screening of medical school applications. METHOD: Using application data and faculty screening outcomes from the 2013 to 2017 application cycles (n = 14,555 applications), the authors created a virtual faculty screener algorithm. A retrospective validation using 2,910 applications from the 2013 to 2017 cycles and a prospective validation using 2,715 applications during the 2018 application cycle were performed. To test the validated algorithm, a randomized trial was performed in the 2019 cycle, with 1,827 eligible applications being reviewed by faculty and 1,873 by algorithm. RESULTS: The retrospective validation yielded area under the receiver operating characteristic (AUROC) values of 0.83, 0.64, and 0.83 and area under the precision-recall curve (AUPRC) values of 0.61, 0.54, and 0.65 for the invite for interview, hold for review, and reject groups, respectively. The prospective validation yielded AUROC values of 0.83, 0.62, and 0.82 and AUPRC values of 0.66, 0.47, and 0.65 for the invite for interview, hold for review, and reject groups, respectively. The randomized trial found no significant differences in overall interview recommendation rates according to faculty or algorithm and among female or underrepresented in medicine applicants. In underrepresented in medicine applicants, there were no significant differences in the rates at which the admissions committee offered an interview (70 of 71 in the faculty reviewer arm and 61 of 65 in the algorithm arm; P = .14). No difference in the rate of the committee agreeing with the recommended interview was found among female applicants (224 of 229 in the faculty reviewer arm and 220 of 227 in the algorithm arm; P = .55). CONCLUSIONS: The virtual faculty screener algorithm successfully replicated faculty screening of medical school applications and may aid in the consistent and reliable review of medical school applicants.

A New Tool for Holistic Residency Application Review: Using Natural Language Processing of Applicant Experiences to Predict Interview Invitation.

PROBLEM: Reviewing residency application narrative components is time intensive and has contributed to nearly half of applications not receiving holistic review. The authors developed a natural language processing (NLP)-based tool to automate review of applicants' narrative experience entries and predict interview invitation. APPROACH: Experience entries (n = 188,500) were extracted from 6,403 residency applications across 3 application cycles (2017-2019) at 1 internal medicine program, combined at the applicant level, and paired with the interview invitation decision (n = 1,224 invitations). NLP identified important words (or word pairs) with term frequency-inverse document frequency, which were used to predict interview invitation using logistic regression with L1 regularization. Terms remaining in the model were analyzed thematically. Logistic regression models were also built using structured application data and a combination of NLP and structured data. Model performance was evaluated on never-before-seen data using area under the receiver operating characteristic and precision-recall curves (AUROC, AUPRC). OUTCOMES: The NLP model had an AUROC of 0.80 (vs chance decision of 0.50) and AUPRC of 0.49 (vs chance decision of 0.19), showing moderate predictive strength. Phrases indicating active leadership, research, or work in social justice and health disparities were associated with interview invitation. The model's detection of these key selection factors demonstrated face validity. Adding structured data to the model significantly improved prediction (AUROC 0.92, AUPRC 0.73), as expected given reliance on such metrics for interview invitation. NEXT STEPS: This model represents a first step in using NLP-based artificial intelligence tools to promote holistic residency application review. The authors are assessing the practical utility of using this model to identify applicants screened out using traditional metrics. Generalizability must be determined through model retraining and evaluation at other programs. Work is ongoing to thwart model "gaming," improve prediction, and remove unwanted biases introduced during model training.

Development and Validation of a Machine Learning Model for Automated Assessment of Resident Clinical Reasoning Documentation.

BACKGROUND: Residents receive infrequent feedback on their clinical reasoning (CR) documentation. While machine learning (ML) and natural language processing (NLP) have been used to assess CR documentation in standardized cases, no studies have described similar use in the clinical environment. OBJECTIVE: The authors developed and validated using Kane's framework a ML model for automated assessment of CR documentation quality in residents' admission notes. DESIGN, PARTICIPANTS, MAIN MEASURES: Internal medicine residents' and subspecialty fellows' admission notes at one medical center from July 2014 to March 2020 were extracted from the electronic health record. Using a validated CR documentation rubric, the authors rated 414 notes for the ML development dataset. Notes were truncated to isolate the relevant portion; an NLP software (cTAKES) extracted disease/disorder named entities and human review generated CR terms. The final model had three input variables and classified notes as demonstrating low- or high-quality CR documentation. The ML model was applied to a retrospective dataset (9591 notes) for human validation and data analysis. Reliability between human and ML ratings was assessed on 205 of these notes with Cohen's kappa. CR documentation quality by post-graduate year (PGY) was evaluated by the Mantel-Haenszel test of trend. KEY RESULTS: The top-performing logistic regression model had an area under the receiver operating characteristic curve of 0.88, a positive predictive value of 0.68, and an accuracy of 0.79. Cohen's kappa was 0.67. Of the 9591 notes, 31.1% demonstrated high-quality CR documentation; quality increased from 27.0% (PGY1) to 31.0% (PGY2) to 39.0% (PGY3) (p < .001 for trend). Validity evidence was collected in each domain of Kane's framework (scoring, generalization, extrapolation, and implications). CONCLUSIONS: The authors developed and validated a high-performing ML model that classifies CR documentation quality in resident admission notes in the clinical environment-a novel application of ML and NLP with many potential use cases.

Development of a Clinical Reasoning Documentation Assessment Tool for Resident and Fellow Admission Notes: a Shared Mental Model for Feedback.

BACKGROUND: Residents and fellows receive little feedback on their clinical reasoning documentation. Barriers include lack of a shared mental model and variability in the reliability and validity of existing assessment tools. Of the existing tools, the IDEA assessment tool includes a robust assessment of clinical reasoning documentation focusing on four elements (interpretive summary, differential diagnosis, explanation of reasoning for lead and alternative diagnoses) but lacks descriptive anchors threatening its reliability. OBJECTIVE: Our goal was to develop a valid and reliable assessment tool for clinical reasoning documentation building off the IDEA assessment tool. DESIGN, PARTICIPANTS, AND MAIN MEASURES: The Revised-IDEA assessment tool was developed by four clinician educators through iterative review of admission notes written by medicine residents and fellows and subsequently piloted with additional faculty to ensure response process validity. A random sample of 252 notes from July 2014 to June 2017 written by 30 trainees across several chief complaints was rated. Three raters rated 20% of the notes to demonstrate internal structure validity. A quality cut-off score was determined using Hofstee standard setting. KEY RESULTS: The Revised-IDEA assessment tool includes the same four domains as the IDEA assessment tool with more detailed descriptive prompts, new Likert scale anchors, and a score range of 0-10. Intraclass correlation was high for the notes rated by three raters, 0.84 (95% CI 0.74-0.90). Scores ≥6 were determined to demonstrate high-quality clinical reasoning documentation. Only 53% of notes (134/252) were high-quality. CONCLUSIONS: The Revised-IDEA assessment tool is reliable and easy to use for feedback on clinical reasoning documentation in resident and fellow admission notes with descriptive anchors that facilitate a shared mental model for feedback.

Association between rectal colonisation by Klebsiella pneumoniae carbapenemase-producing K. pneumoniae and mortality: a prospective, observational study.

OBJECTIVES: We evaluated the association of Klebsiella pneumoniae carbapenemase-producing K. pneumoniae (KPC-Kp) rectal colonisation with crude mortality and whether this association is independent of the risk of KPC-Kp infection. METHODS: This was a prospective cohort study of patients followed-up 90 days after a study of rectal colonisation. Cox regression was used to study the variables associated with crude mortality. Sensitivity analyses for 90-day crude mortality in different subcohorts were performed. RESULTS: A total of 1244 patients (1078 non-colonised and 166 colonised) were included. None of the non-colonised patients and 78 (47.0%) of the colonised patients developed KPC-Kp infection. The 90-day crude mortality was 18.0% (194/1078) in non-colonised patients and 41.6% (69/166) in colonised patients. Rectal colonisation was not associated with crude mortality [hazard ratio (HR) = 1.03, 95% confidence interval (CI) 0.69-1.54; P = 0.85] when the model was adjusted for severe KPC-Kp infection [INCREMENT-CPE score (ICS) > 7]. KPC-Kp infection with ICS > 7 was associated with an increased risk of all-cause mortality (HR = 2.21, 95% CI 1.35-3.63; P = 0.002). In the sensitivity analyses, KPC-Kp colonisation was not associated with mortality in any of the analysed subcohorts, including patients who did not develop KPC-Kp infection (HR = 0.93, 95% CI 0.60-1.43; P = 0.74). CONCLUSION: KPC-Kp rectal colonisation was not associated with crude mortality. Mortality increased when colonised patients developed severe KPC-Kp infection (ICS > 7). Rectal colonisation was a necessary although insufficient condition to die from a KPC-Kp infection.

Sustainable Functionalization of PAN to Improve Tinctorial Capacity.

This study may open a new way to obtain the coloration of a polymer during functionalization. Two polyacrylonitrile (PAN) polymers in the form of textile fibers (Melana and Dralon L) were subjected to functionalization treatments in order to improve the dyeing capacity. The functionalizations determined by an organo-hypervalent iodine reagent developed in situ led to fiber coloration without using dyes. KIO3 was formed in situ from the interaction of aqueous solutions of 3-9% KOH with 3-9% I2, at 120 °C. The yellow-orange coloration appeared as a result of the transformations in the chemical structure of each functionalized polymer, with the formation of iodinehydrin groups. The degree of functionalization directly influenced the obtained color. The results of the Fourier Transform Infrared Spectroscopy (FTIR), Scanning Electron Microscopy (SEM), Energy Dispersive X-ray Spectroscopy (EDX), Map and Temogravimetric Analysis (TG) plus Differential Thermal (DTA) analyses indicated the presence of new functional groups, such as iodine-oxime. The X-ray diffraction (XRD) analysis confirmed the change of the crystalline/amorphous ratio in favor of the former. The new groups introduced by functionalization make it possible to dye with classes of dyes specific to these groups, but not specific to PAN fibers, thus improving their dyeing capacity.

Development and Validation of a Machine Learning-Based Decision Support Tool for Residency Applicant Screening and Review.

PURPOSE: Residency programs face overwhelming numbers of residency applications, limiting holistic review. Artificial intelligence techniques have been proposed to address this challenge but have not been created. Here, a multidisciplinary team sought to develop and validate a machine learning (ML)-based decision support tool (DST) for residency applicant screening and review. METHOD: Categorical applicant data from the 2018, 2019, and 2020 residency application cycles (n = 8,243 applicants) at one large internal medicine residency program were downloaded from the Electronic Residency Application Service and linked to the outcome measure: interview invitation by human reviewers (n = 1,235 invites). An ML model using gradient boosting was designed using training data (80% of applicants) with over 60 applicant features (e.g., demographics, experiences, academic metrics). Model performance was validated on held-out data (20% of applicants). Sensitivity analysis was conducted without United States Medical Licensing Examination (USMLE) scores. An interactive DST incorporating the ML model was designed and deployed that provided applicant- and cohort-level visualizations. RESULTS: The ML model areas under the receiver operating characteristic and precision recall curves were 0.95 and 0.76, respectively; these changed to 0.94 and 0.72, respectively, with removal of USMLE scores. Applicants' medical school information was an important driver of predictions-which had face validity based on the local selection process-but numerous predictors contributed. Program directors used the DST in the 2021 application cycle to select 20 applicants for interview that had been initially screened out during human review. CONCLUSIONS: The authors developed and validated an ML algorithm for predicting residency interview offers from numerous application elements with high performance-even when USMLE scores were removed. Model deployment in a DST highlighted its potential for screening candidates and helped quantify and mitigate biases existing in the selection process. Further work will incorporate unstructured textual data through natural language processing methods.

Medical student in the family health strategy o the first years of college: perception of graduates / Estudiante de medicina en la estrategia salud de la familia en series iniciiales: percepción de los egresos / Estudante de medicina na estratégia saúde da família em séries iniciais: percepção dos egressos

There is a lack of knowledge about the effective value of the experience gained by medical students who participate in the Family Health Strategy (Estratégia Saúde da Família (ESF)) during the early stages of their medical training. This teaching strategy is based on learning by experiencing the problems that exist in real life. This study proposed to understand the value of this teaching strategy from the viewpoint of the students who had participated, after their graduation. The method adopted was a qualitative study conducted through interviews with students who graduated in the years 2009, 2010 and 2011. The data analysis used the hermeneutic dialectic technique as its model. The graduates considered that this experience enabled them to understand the organization and functioning of the health service and the context of the daily life of the users. This experience facilitated the doctor patient relationship, the development of clinical reasoning and the bond with the user. However the students emphasized that a lack of maturity prevented them gaining a higher level of benefit from the experience. Therefore, although the structure of the course is permeated by advances and challenges, it was concluded that this experience contributed to the student's learning of certain essential elements of medical training.

Considerando el desconocimiento del efectivo significado de la vivencia de estudiantes de medicina al ser inserido Estrategia Salud de la Familia (ESF) en series iniciales del curso por medio de estrategias de enseñanza basadas en la problematización de la realidad, se propuso a comprender tal inserción en la óptica de los egresos. Estudio cualitativo realizado por medio de entrevistas con egresos formados en los años de 2009, 2010 y 2011. El análisis de los datos tuvo como referencia la técnica de la hermenéutica-dialéctica. Los egresos consideran que esa inserción posibilitó el conocimiento de la organización y funcionamiento del servicio de salud y del contexto de vida de los usuarios, facilitó la relación médico paciente, el desarrollo del raciocinio clínico y el vínculo. Destacan, aunque, que la inmadurez del estudiante impide mayor aprovechamiento de la vivencia. Sin embargo esa trayectoria esté permeada por avances y desafíos, se concluye que ella se muestra capaz de sedimentar elementos imprescindibles a la formación médica.

Considerando o desconhecimento do efetivo significado da vivência de estudantes de medicina ao serem inserido Estratégia Saúde da Família (ESF) em séries iniciais do curso por meio de estratégias de ensino baseadas na problematizaçao da realidade, propôs-se a compreender tal inserção na ótica dos egressos. Estudo qualitativo realizado por meio de entrevistas com egressos formados nos anos de 2009, 2010 e 2011. A análise dos dados teve como referência a técnica da hermenêutica-dialética. Os egressos consideram que essa inserção possibilitou o conhecimento da organização e funcionamento do serviço de saúde e do contexto de vida dos usuários, facilitou a relação médico paciente, o desenvolvimento do raciocínio clínico e o vínculo. Destacam, no entanto, que a imaturidade do estudante impede maior aproveitamento da vivência. Embora essa trajetória esteja permeada por avanços e desafios, conclui-se que ela se mostra capaz de sedimentar elementos imprescindíveis à formação médica.

Medical student in the family health strategy on the first years of college: perception of graduates.

There is a lack of knowledge about the effective value of the experience gained by medical students who participate in the Family Health Strategy (Estratégia Saúde da Família (ESF)) during the early stages of their medical training. This teaching strategy is based on learning by experiencing the problems that exist in real life. This study proposed to understand the value of this teaching strategy from the viewpoint of the students who had participated, after their graduation. The method adopted was a qualitative study conducted through interviews with students who graduated in the years 2009, 2010 and 2011. The data analysis used the hermeneutic dialectic technique as its model. The graduates considered that this experience enabled them to understand the organization and functioning of the health service and the context of the daily life of the users. This experience facilitated the doctor patient relationship, the development of clinical reasoning and the bond with the user. However the students emphasized that a lack of maturity prevented them gaining a higher level of benefit from the experience. Therefore, although the structure of the course is permeated by advances and challenges, it was concluded that this experience contributed to the student's learning of certain essential elements of medical training.

Deceased-donor kidney transplantation: improvement in long-term survival.

BACKGROUND: Despite marked improvement in short-term renal allograft survival rates (GSR) in recent years, improvement in long-term GSR remained elusive. METHODS: We analysed the kidney transplant experience at our centre accrued over four decades to evaluate how short-term and long-term GSR had changed and to identify risk factors affecting graft survival. The study included 1476 adult recipients of a deceased-donor kidney transplant who were transplanted between 1963 and 2006 and who had received one of five distinct immunosuppressive protocols. RESULTS: Five-year actual GSR steadily improved over the years as immunosuppressive therapy evolved (22-86%, P < 0.001) in spite of an increasing trend in the transplantation of higher-risk donor-recipient pairings. For those whose grafts functioned for the first year, subsequent 4-year GSR (5-year conditional GSR) also improved significantly (63-92%, P < 0.001). Acute rejection and delayed graft function (DGF) were the most significant risk factors for actual graft survival, while acute rejection was the only significant risk factor for conditional GSR. Use of kidneys from expanded-criteria donors (ECD) was not a risk factor, compared to the use of standard-criteria donor kidneys for either 5-year actual or conditional GSR. There was an impressive decline in the incidence of acute rejection events (77.4-5.8%, P < 0.001). While the DGF rate had decreased, it still remained high (68.7-38.5%, P < 0.001). CONCLUSIONS: We found a significant improvement in both short-term and long-term GSR of deceased-donor kidney transplants over the last four decades. These improvements are most likely related to the decreased incidence of acute rejection episodes. Minimizing acute rejection events and preventing DGF could result in further improvement in the GSR. Our experience in the judicious use of ECD kidneys suggests that this source of kidneys could be expanded further.

Enfermedad tromboembólica venosa y olanzapina / Venous thromboembolic disease and olanzapine

No disponible

[Severe mitral regurgitation in Libman-Sacks endocarditis. Conservative surgery]. / Regurgitación mitral severa en endocarditis de Libman-Sacks. Cirugía reparadora.

Libman-Sacks endocarditis is a classic but rarely symptomatic manifestation of systemic lupus erythematosus, and valvular surgery is needed in a few cases. We present a patient with systemic lupus erythematosus and Libman-Sacks endocarditis that progressed rapidly to severe mitral regurgitation that needed surgery; surgical valve repair was decided upon. The literature on this topic is reviewed.

Regurgitación mitral severa en endocarditis de Libman-Sacks. Cirugía reparadora / Severe mitral regurgitation in Libman-Sacks endocarditis. Conservative surgery

La endocarditis de Libman-Sacks es la afección cardíaca más clásica del lupus eritematoso, pero la afectación clínica es poco frecuente, por lo que la cirugía valvular es necesaria en pocas ocasiones. Presentamos un caso de lupus eritematoso con endocarditis de Libman-Sacks que evolucionó rápidamente a regurgitación mitral severa con necesidad de cirugía, por lo que se optó por la reparación valvular. Se hace una revisión de la literatura médica del tema (AU)

Libman-Sacks endocarditis is a classic but rarely symptomatic manifestation of systemic lupus erythematosus, and valvular surgery is needed in a few cases. We present a patient with systemic lupus erythematosus and Libman-Sacks endocarditis that progressed rapidly to severe mitral regurgitation that needed surgery; surgical valve repair was decided upon. The literature on this topic is reviewed (AU)