Medical Knowledge Assessment by Hematology and Medical Oncology In-Training Examinations Are Better Than Program Director Assessments at Predicting Subspecialty Certification Examination Performance.

The Accreditation Council for Graduate Medical Education's Next Accreditation System requires training programs to demonstrate that fellows are achieving competence in medical knowledge (MK), as part of a global assessment of clinical competency. Passing American Board of Internal Medicine (ABIM) certification examinations is recognized as a metric of MK competency. This study examines several in-training MK assessment approaches and their ability to predict performance on the ABIM Hematology or Medical Oncology Certification Examinations. Results of a Hematology In-Service Examination (ISE) and an Oncology In-Training Examination (ITE), program director (PD) ratings, demographic variables, United States Medical Licensing Examination (USMLE), and ABIM Internal Medicine (IM) Certification Examination were compared. Stepwise multiple regression and logistic regression analyses evaluated these assessment approaches as predictors of performance on the Hematology or Medical Oncology Certification Examinations. Hematology ISE scores were the strongest predictor of Hematology Certification Examination scores (ß = 0.41) (passing odds ratio [OR], 1.012; 95 % confidence interval [CI], 1.008-1.015), and the Oncology ITE scores were the strongest predictor of Medical Oncology Certification Examination scores (ß = 0.45) (passing OR, 1.013; 95 % CI, 1.011-1.016). PD rating of MK was the weakest predictor of Medical Oncology Certification Examination scores (ß = 0.07) and was not significantly predictive of Hematology Certification Examination scores. Hematology and Oncology ITEs are better predictors of certification examination performance than PD ratings of MK, reinforcing the effectiveness of ITEs for competency-based assessment of MK.

Prevalence and Correlates of Prohibited Questions in Internal Medicine and Pediatrics Fellowship Interviews During One Appointment Year.

PURPOSE: This study sought to investigate how frequently applicants to internal medicine (IM) and pediatrics fellowships are subjected to prohibited questions, how correlates of these interview questions compare between IM and pediatrics fellowship applicants, and which applicant subgroups are most affected. METHOD: The National Resident Matching Program (NRMP) emailed an anonymous survey to all applicants for the 2021 appointment year to the Medical Specialties Matching Program (i.e., IM fellowship Matches) and Pediatric Specialties Fellowship Match who certified rank order lists (ROLs). The survey addressed specific questions regarding the use of legally prohibited questions and questions that violate the NRMP's Match Participation Agreement during interview-related activities. Experiences of respondents were compared by preferred subspecialty and respondent demographics within IM and pediatrics. RESULTS: The final response rates of IM and pediatrics fellowship applicants who certified ROLs, including complete and partial surveys, were 21.7% (1,483/6,847) and 23.4% (385/1,648), respectively. Of the IM and pediatrics respondents, 432/1,296 (33.3%) and 97/366 (26.5%), respectively, reported being asked at least one prohibited demographic question. The most commonly asked prohibited questions pertained to relationship or marital status (IM: 312/1,296, 24.1%; pediatrics: 69/367, 18.8%), national origin (IM: 200/1,296, 15.4%; pediatrics: 30/365, 8.2%), and family planning (IM: 104/1,288, 8.1%; pediatrics: 14/366, 3.8%). Nearly 25% of IM and pediatrics respondents reported being asked to identify other programs they applied to or interviewed with. Most often, these questions came from program faculty (IM: 238/303, 78.5%; pediatrics: 69/88, 78.4%) or program directors (IM: 84/303, 27.7%; pediatrics: 18/88, 20.5%). CONCLUSIONS: Substantial proportions of IM and pediatrics fellowship applicants reported being asked prohibited questions during fellowship interview-related activities. Additional educational efforts are needed to eradicate such questions from the interview process.

Perspectives on the quality and utility of letters conforming to the AAIM guidelines.

Background Standardized letters of recommendation (SLOR) have become common features of the medical school to residency transition. Research has shown many advantages over the narrative letter of recommendation including improved letter-writing efficiency, ease of interpretation, and improved reliability as performance predictors. Currently, at least four specialties require fellowship SLORs. Internal medicine adopted its SLOR in 2017. Previous research showed fellowship program directors' satisfaction with the 2017 guidelines. Little is known about residency program directors' acceptance and adherence to the guidelines. Objectives The study sought to assess the adoption rate of each component, barriers to adoption, time commitment, and alignment with intended goals of the guidelines. Methods Anonymous survey links were posted to an internal medicine discussion forum prior to the guidelines in spring 2017 and twice following the guidelines in fall 2018 and winter 2019. Two-sample tests of proportions were used to compare respondent characteristics with known survey population data. Pre- and post-survey comparisons were assessed for statistical significance with Pearson chi-squared statistic. Results The response rate varied from 30% to 35% for each survey period. Medical knowledge, patient care, interpersonal and communication skills, professionalism, and scholarly activity were reported frequently (>96%) at baseline. Inclusion of residency program characteristics, systems-based practice, practice-based learning and improvement, and skills sought to master increased over the study period. Conclusions The new guidelines improved uniform reporting of all core competency data. Overall, the gains were modest, as many pre-survey respondents reported high rates of including components within the guidelines.

The American Society of Hematology and ASCO Curricular Milestones for Assessment of Fellows in Hematology/Oncology: Development, Reflection, and Next Steps.

The American Society of Hematology (ASH)/ASCO Curricular Milestones is a tool for assessment and teaching for fellows in hematology/oncology. The expectations of the Next Accreditation System of the Accreditation Council of Graduate Medical Education (ACGME) was developed over years from the creation of the six core competencies in 1999 to the current data-driven outcomes-based system. The current internal medicine subspecialty milestones (ACGME reporting milestones) follow the general rubric of the general internal medicine milestones. The ASH/ASCO curricular milestones were developed from the foundational elements of the specialty, and they are interwoven with the ACGME reporting milestones. The 2017 ACGME Milestones Report shows that the milestones display progression in performance through clear anchors. Educational outcomes are available in many specialties. The internal medicine subspecialties have been given the opportunity to update the ACGME reporting milestones. The ACGME has acknowledged that these milestones may be different for each of the specialties. The program committees of ASH and ASCO agree that revision of the ACGME reporting milestones would decrease the overlap of domains, lack of clarity, and negative language that is present in version 1.0. ASH and ASCO are working with the ACGME and American Board of Internal Medicine (ABIM) to develop Curricular Milestones, version 2.0.

Problematic communications during 2016 fellowship recruitment in internal medicine.

Some internal medicine residency program directors have expressed concerns that their third-year residents may have been subjected to inappropriate communication during the 2016 fellowship recruitment season. The authors sought to study applicants' interpersonal communication experiences with fellowship programs. Many respondents indicated that they had been asked questions that would constitute violations of the National Residency Matching Program (NRMP) Communications Code of Conduct agreement, including how they plan to rank specific programs. Moreover, female respondents were more likely to have been asked questions during interview experiences about other programs to which they applied, and about their family plans. Post-interview communication policies were not made clear to most applicants. These results suggest ongoing challenges for the internal medicine community to improve communication with applicants and uniform compliance with the NRMP communications code of conduct during the fellowship recruitment process.

Studying the effects of ACGME duty hours limits on resident satisfaction: results from VA learners' perceptions survey.

BACKGROUND: As the Accreditation Council on Graduate Medical Education (ACGME) deliberates over further limiting duty hours of graduate medical education (GME) trainees, few large-scale studies have shown residents to be satisfied with the effect the 2003 standards have had on clinical care, education outcomes, or working environments. This study measures the effect of the 2003 duty hours limits on resident-reported satisfaction with GME training during their rotations through the Department of Veterans Affairs (VA) medical centers from 2001 through 2007. METHOD: Self-reported satisfaction with clinical care and education environments were assessed by comparing responses to VA's annual Learners' Perceptions Survey administered before 2003 with responses administered after 2003. To measure duty hours effects on satisfaction, before-after differences were adjusted for covariate biases modeled after an exhaustive covariate search with 10-fold cross-validation. Because nonteaching controls are not available in satisfaction studies, we used a robust differencing variable technique to control before-after differences for trend biases in the simultaneous presence of missing data and possible model misspecification. RESULTS: There were 19,605 responders. Adjusting for covariate and trend biases, after the 2003 ACGME standards, 25% more residents in medicine specialties reported satisfaction with VA clinical environment and 11% more with VA preceptors and faculty. For surgery, 33% more residents reported satisfaction with VA clinical environment and 12% more with VA preceptors and faculty. Satisfaction with working environment was mixed. CONCLUSIONS: The 2003 ACGME duty hours standards were associated with improved satisfaction for resident clinical training and learning environments.

Measuring progressive independence with the resident supervision index: theoretical approach.

BACKGROUND: Graduate medical education is based on an on-the-job training model in which residents provide clinical care under supervision. The traditional method is to offer residents graduated levels of responsibility that will prepare them for independent practice. However, if progressive independence from supervision exceeds residents' progressive professional development, patient outcomes may be at risk. Leaders in graduate medical education have called for "optimal" supervision, yet few studies have conceptually defined what optimal supervision means and whether optimal care is theoretically compatible with progressive independence, nor have they developed a test for progressive independence. OBJECTIVE: This research develops theory and analytic models as part of the Resident Supervision Index to quantify the intensity of supervision. METHODS: We introduce an explicit set of assumptions for an ideal patient-centered theory of optimal supervision of resident-provided care. A critical assumption is that informed attending staff will use available resources to optimize patient outcomes first and foremost, with residents gaining clinical competencies by contributing to optimal care. Next, we derive mathematically the consequences of these assumptions as theoretical results. RESULTS: Under optimal supervision, (1) patient outcome is expected to be no worse than if residents were not involved, (2) supervisors will avoid undersupervising residents (when patients are at increased risk for poor outcomes) or oversupervising residents (when residents miss clinical opportunities to practice care), (3) optimal patient outcomes will be compatible with progressive independence, (4) progressive development can be inferred from progressive independence whenever residents contribute to patient care, and (5) analytic models that test for progressive independence will emphasize adjusting the association between length of graduate medical education training and supervision for case complexity and clinic workload, but not patient health outcomes. CONCLUSION: An explicit theoretical framework is critical to measure scientifically progressive independence from supervision using graduate medical education data.

Measuring progressive independence with the resident supervision index: empirical approach.

BACKGROUND: A Resident Supervision Index (RSI) developed by our research team quantifies the intensity of resident supervision in graduate medical education, with the goal of testing for progressive independence. The 4-part RSI method includes a survey instrument for staff and residents (RSI Inventory), a strategy to score survey responses, a theoretical framework (patient centered optimal supervision), and a statistical model that accounts for the presence or absence of supervision and the intensity of patient care. METHODS: The RSI Inventory data came from 140 outpatient encounters involving 57 residents and 37 attending physicians during a 3-month period at a Department of Veterans Affairs outpatient clinic. Responses are scored to quantitatively measure the intensity of resident supervision across 10 levels of patient services (staff is absent, is present, participated, or provided care with or without a resident), case discussion (resident-staff interaction), and oversight (staff reviewed case, reviewed medical chart, consulted with staff, or assessed patient). Scores are analyzed by level and for patient care using a 2-part model (supervision initiated [yes or no] versus intensity once supervision was initiated). RESULTS: All resident encounters had patient care supervision, resident oversight, or both. Consistent with the progressive independence hypothesis, residents were 1.72 (P  =  .019) times more likely to be fully responsible for patient care with each additional postgraduate year. Decreasing case complexity, increasing clinic workload, and advanced nonmedical degrees among attending staff were negatively associated with supervision intensity, although associations varied by supervision level. CONCLUSIONS: These data are consistent with the progressive independence hypothesis in graduate medical education and offer empirical support for the 4-part RSI method to quantify the intensity of resident supervision for research, program evaluation, and resident assessment purposes. Before informing policy, however, more scientific research in actual teaching settings is needed to better understand the relationships among patient outcomes, clinic workload, case complexity, and graduate medical education experience in resident supervision and professional development.

Measuring the intensity of resident supervision in the department of veterans affairs: the resident supervision index.

PURPOSE: To develop a survey instrument designed to quantify supervision by attending physicians in nonprocedural care and to assess the instrument's feasibility and reliability. METHOD: In 2008, the Department of Veterans Affairs (VA) Office of Academic Affiliations convened an expert panel to adopt a working definition of attending supervision in nonprocedural patient care and to construct a survey to quantify it. Feasibility was field-tested on residents and their supervising attending physicians at primary care internal medicine clinics at the VA Loma Linda Healthcare System in their encounters with randomly selected outpatients diagnosed with either major depressive disorder or diabetes. The authors assessed both interrater concurrent reliability and test-retest reliability. RESULTS: The expert panel adopted the VA's definition of resident supervision and developed the Resident Supervision Index (RSI) to measure supervision in terms of residents' case understanding, attending physicians' contributions to patient care through feedback to the resident, and attending physicians' time (minutes). The RSI was field-tested on 60 residents and 37 attending physicians for 148 supervision episodes from 143 patient encounters. Consent rates were 94% for residents and 97% for attending physicians; test-retest reliability intraclass correlations (ICCs) were 0.93 and 0.88, respectively. Concurrent reliability between residents' and attending physicians' reported time was an ICC of 0.69. CONCLUSIONS: The RSI is a feasible and reliable measure of resident supervision that is intended for research studies in graduate medical education focusing on education outcomes, as well as studies assessing quality of care, patient health outcomes, care costs, and clinical workload.

Expression of DeltaF508 Cystic Fibrosis Transmembrane Regulator (CFTR) Decreases Membrane Sialylation.

Chronic colonization and infection of the lung with Pseudomonas aeruginosa is a major cause of morbidity and mortality in cystic fibrosis (CF) patients. Imundo, et al. determined that CF cells had a higher concentration of an asialoganglioside (asialo-G(M1)), to which both P. aeruginosa and S. aureus bound preferentially. We sought to determine if the expression of mutant CFTR is associated with altered sialylation. Our study of epithelial cells transfected with normal and mutant DeltaF508 CFTR, the defect in the majority of CF patients in the United States, were analyzed by ELISA and FACS analysis of cell membranes labeled with lectins which bind to Neu5Ac. We determined that DeltaF508 CFTR is associated with decreased membrane sialic acid residues in the alpha2, 3 position and increased concentrations of asialo- G(M1). Quantitation of sialic acids released from the cellular membranes demonstrated that the presence of the DeltaF508 CFTR is associated with markedly decreased membrane sialylation, but similar cytoplasmic sialylation. Thus, DeltaF508 defect is correlated with decreased expression of G(M1) and with decreased sialylation of all cell surface structures, and this change occurs during post-translational modification of glycoproteins and glycolipids. This may be one factor involved in the chronic bacterial colonization seen in these patients.