Fellowship-trained physicians who let their geriatric medicine certification lapse: A national survey.

BACKGROUND: Only 62.6% of fellowship-trained and American Board of Internal Medicine (ABIM)-certified geriatricians maintain their specialty certification in geriatric medicine, the lowest rate among all internal medicine subspecialties and the only subspecialty in which physicians maintain their internal medicine certification at higher rates than their specialty certification. This study aims to better understand underlying issues related to the low rate of maintaining geriatric medicine certification in order to inform geriatric workforce development strategies. METHODS: Eighteen-item online survey of internists who completed a geriatric medicine fellowship, earned initial ABIM certification in geriatric medicine between 1999 and 2009, and maintained certification in internal medicine (and/or another specialty but not geriatric medicine). Survey domains: demographics, issues related to maintaining geriatric medicine certification, professional identity, and current professional duties. RESULTS: 153/723 eligible completed surveys (21.5% response). Top reasons for not maintaining geriatric medicine certification were time (56%), cost of maintenance of certification (MOC) (45%), low Medicare reimbursement for geriatricians' work (32%), and no employer requirement to maintain geriatric medicine certification (31%). Though not maintaining geriatric medicine certification, 68% reported engaging in professional activities related to geriatric medicine. Reflecting on career decisions, 56% would again complete geriatric medicine fellowship, 21% would not, and 23% were unsure. 54% considered recertifying in geriatric medicine. 49% reported flexible MOC assessment options would increase likelihood of maintaining certification. CONCLUSIONS: The value proposition of geriatric medicine certification needs strengthening. Geriatric medicine leaders must develop strategies and tactics to reduce attrition of geriatricians by enhancing the value of geriatric medicine expertise to key stakeholders.

Changes in Stress and Workplace Shortages Reported by U.S. Critical Care Physicians Treating Coronavirus Disease 2019 Patients.

OBJECTIVES: Eleven months into the coronavirus disease 2019 pandemic, the country faces accelerating rates of infections, hospitalizations, and deaths. Little is known about the experiences of critical care physicians caring for the sickest coronavirus disease 2019 patients. Our goal is to understand how high stress levels and shortages faced by these physicians during Spring 2020 have evolved. DESIGN: We surveyed (October 23, 2020 to November 16, 2020) U.S. critical care physicians treating coronavirus disease 2019 patients who participated in a National survey earlier in the pandemic (April 23, 2020 to May 3, 2020) regarding their stress and shortages they faced. SETTING: ICU. PATIENTS: Coronavirus disease 2019 patients. INTERVENTION: Irrelevant. MEASUREMENT: Physician emotional distress/physical exhaustion: low (not at all/not much), moderate, or high (a lot/extreme). Shortage indicators: insufficient ICU-trained staff and shortages in medication, equipment, or personal protective equipment requiring protocol changes. MAIN RESULTS: Of 2,375 U.S. critical care attending physicians who responded to the initial survey, we received responses from 1,356 (57.1% response rate), 97% of whom (1,278) recently treated coronavirus disease 2019 patients. Two thirds of physicians (67.6% [864]) reported moderate or high levels of emotional distress in the Spring versus 50.7% (763) in the Fall. Reports of staffing shortages persisted with 46.5% of Fall respondents (594) reporting a staff shortage versus 48.3% (617) in the Spring. Meaningful shortages of medication and equipment reported in the Spring were largely alleviated. Although personal protective equipment shortages declined by half, they remained substantial. CONCLUSIONS: Stress, staffing, and, to a lesser degree, personal protective equipment shortages faced by U.S. critical care physicians remain high. Stress levels were higher among women. Considering the persistence of these findings, rising levels of infection nationally raise concerns about the capacity of the U.S. critical care system to meet ongoing and future demands.

Opportunities for Maintenance of Certification to Better Reflect Scope of Practice Among Medical Oncologists.

PURPOSE: Critics argue that the American Board of Internal Medicine's medical oncology Maintenance of Certification examination requires medical oncologists with a narrow scope of practice to spend time studying material that is no longer relevant to their practice. However, no data are available describing the scope of practice for medical oncologists. METHODS: Using Medicare claims, we examined the scope of practice for 9,985 medical oncologists who saw 8.6 million oncology conditions in 2016, each of which was assigned to 1 of 23 different condition groups. Scope of practice was then measured as the percentage of oncology conditions within each of the 23 groups. We grouped physicians with similar scopes of practice by applying K-means clustering to the percentage of conditions seen. The scope of practice for each physician cluster was determined from the cancers that encompassed the majority of average oncology conditions seen among physicians composing the cluster. RESULTS: We found 20 distinct scope-of-practice clusters. The largest (n = 6,479 [65.5%]) had a general oncology scope of practice. The remaining physicians focused on a narrow scope of cancers, including 22.6% focused on ≥ 1 solid tumors and 11.9% focused on hematologic malignancies. The largest focused cluster accounted for 7.7% of physicians focused on breast cancer. CONCLUSION: A single American Board of Internal Medicine Maintenance of Certification assessment in medical oncology is most appropriate for approximately 65% of certified medical oncologists' practices. However, the addition of assessments focused on breast cancer and hematologic malignancies could increase this figure to upwards of 85% of certified medical oncologists.

"That Was Pretty Powerful": a Qualitative Study of What Physicians Learn When Preparing for Their Maintenance-of-Certification Exams.

BACKGROUND: A key component of Maintenance of Certification (MOC) for family and internal medicine physicians is the requirement to pass a periodic examination of medical knowledge. Little is known about the effects of preparing for MOC exams on knowledge and practice. OBJECTIVE: To understand how MOC exam preparation can affect knowledge and practice. DESIGN: Qualitative, semi-structured interviews, 45-60 min each, conducted by telephone at participants' convenience. PARTICIPANTS: A total of 80 primary care physicians from the American Board of Family Medicine (ABFM) and the American Board of Internal Medicine (ABIM) who had recently taken an MOC exam; the sample purposefully selected to represent diversity of experiences with MOC exams and range of opinions about MOC, as well as diversity of participant backgrounds-gender, practice type, etc. APPROACH: Close analysis of physicians' accounts of what they learned when preparing for an MOC exam and how this knowledge affected their practice. RESULTS: Sixty-seven of 80 physicians stated they gained knowledge relevant to their practice. Sixty-three gave concrete examples of how this affected their practice, including direct changes to patient care (e.g., improved diagnosis or prescribing and reduced unnecessary testing) or less direct changes (e.g., improved ability to co-manage with other providers or communicate with patients). Physicians also described sharing what they learned with others, including peers and trainees. LIMITATIONS: Interviews could have been affected by recall and/or social desirability bias, as well as researchers' role as board staff. Although we followed a recruitment protocol designed to mitigate recruitment acceptance bias, our findings may not be generalizable to wider groups of physicians. CONCLUSIONS: Most physicians from two primary care specialties interviewed reported ways in which studying for an MOC exam resulted in acquiring knowledge that was both relevant and beneficial to their patient care.

Tiny Moments Matter: Promoting Professionalism in Everyday Practice.

INTRODUCTION: Professionalism rests upon a number of individual, environmental, and societal level factors, leading to specific professional behavior in specific situations. Focusing on professional lapses to identify and remediate unprofessional physicians is incomplete. We explored professionalism in practicing internal medicine physicians in the context of everyday practice, to highlight how typical experiences contribute to positive, yet often unnoticed, professional behavior. METHODS: In-depth interviews were used to uncover 13 physicians' most meaningful experiences of professionalism. Data were collected and analyzed using a grounded theory approach. RESULTS: Results revealed several themes around which physicians embody professionalism in their daily lives. Physicians feel most professional when they are able to connect and establish trust with patients and colleagues and when they serve as positive role models to others. Physicians conceptualize professionalism as a dynamic and evolving competency, one that requires a lifelong commitment and that provides opportunities for lifelong learning. DISCUSSION: Focusing on actual perceptions of experiences in practice offers important insights into how physicians think about professionalism beyond a traditional remediation and lapses perspective. Physicians often go out of their way to connect with patients and colleagues, serving and modeling for others, often at the expense of their own work-life balance. These moments help to infuse energy and positivity into physician practices during a time when physicians may feel overburdened, overscheduled, and overregulated. Understanding professionalism as developmental helps frame professionalism as a lifelong competency subject to growth and modification over time.

Co-creation by the ABIM Geriatric Medicine Board and the AGS - Helping Move Geriatrics Forward.

The American board of internal medicine (ABIM) establishes standards for physicians. The American geriatrics society (AGS) is a not-for-profit membership organization of nearly 6,000 health professionals devoted to improving the health, independence, and quality of life of all older people. Beginning in 2013, ABIM redesigned its governance structure, including the role of the specialty boards. Specialty boards are charged with responsibilities for oversight in four main areas: (1) the assessments used in initial certification and maintenance of certification (MOC); (2) medical knowledge self-assessment and practice assessment in the specialty; (3) building relationships with relevant professional societies and other organizational stakeholders; and (4) issues related to training requirements for initial certification eligibility within the specialty. The aim of this paper is to inform the geriatrics community regarding the function of geriatric medicine board (GMB) of the ABIM, and to invite the geriatrics community to fully engage with and leverage the GMB as a partner to: (1) develop better certification examinations and processes, identifying better knowledge and practice assessments, and in establishing appropriate training and MOC requirements for geriatric medicine; (2) leverage ABIM assets to conduct applied research to guide the field in the areas of training and certification and workforce development in geriatric medicine; (3) make MOC relevant for practicing geriatricians. Active engagement of the geriatrics community with ABIM and the GMB will ensure that certification in geriatric medicine provides the greatest possible value and meaning to physicians, patients, and the public.

How hospitalists work to pull healthcare teams together.

PURPOSE: The purpose of this paper is to document everyday practices by which hospitalist physicians negotiate barriers to effective teamwork. DESIGN/METHODOLOGY/APPROACH: Ethnographic observation with a sample of hospitalists chosen to represent a range of hospital and practice types. FINDINGS: Hospitals rely on effective, interprofessional teamwork but typically do not support it. Hospitalist physicians must bridge the internal boundaries within their hospitals to coordinate their patients' care, but they face challenges - scattered patients, fragmented information, uncoordinated teams, and unreliable processes - that can impact the timeliness and safety of care. Hospitalists largely rely on personal presence and memory to deal with these challenges. Some invent low-tech supports for teamwork, but these are typically neither tested nor shared with others. Formal support for teamwork, primarily case management rounds, is applied unevenly and may not be respected by all team members. RESEARCH LIMITATIONS/IMPLICATIONS: The findings are drawn from observation over a limited period of time with a small, purposefully chosen sample of physicians and hospitals. Practical implications - Hospitals must recognize the issues hospitalists and other providers face, evaluate and disseminate supports for teamwork, and make interprofessional teamwork a core feature of hospital design and evaluation. ORIGINALITY/VALUE: The authors show the nuances of how hospitalists struggle to practice teamwork in a challenging context, and how the approaches they take (relying on memory and personal presence) do not address, and may actually contribute to, the system-level problems they face.

Assessing the Quality of Osteoporosis Care in Practice.

BACKGROUND: Patients with osteoporosis can sustain fractures following falls or other minimal trauma. This risk of fracture can be reduced through appropriate diagnostic testing, pharmacologic therapy, and other readily measured standards of care. OBJECTIVES: Our aim was to develop a credible clinical performance assessment to measure physicians' quality of osteoporosis care, and determine reasonable performance standards for both competent and excellent care. DESIGN: This was a retrospective cohort study. PARTICIPANTS: Three hundred and eighty one general internists and subspecialists with time-limited board certification were included in the study. MAIN MEASURES: Performance rates on eight evidence-based measures were obtained from the American Board of Internal Medicine (ABIM) Osteoporosis Practice Improvement Module® (PIM), a web-based tool that uses medical chart reviews to help physicians assess and improve care. We applied a patented methodology, using an adaptation of the Angoff standard-setting method and the Dunn-Rankin method, with an expert panel skilled in osteoporosis care to form a composite and establish standards for both competent and excellent care. Physician and practice characteristics, including a practice infrastructure score based on the Physician Practice Connections Readiness Survey (PPC-RS), were used to examine the validity of the inferences made from the composite scores. KEY RESULTS: The mean composite score was 67.54 out of 100 maximum points with a reliability of 0.92. The standard for competent care was 46.87, and for excellent care it was 83.58. Both standards had high classification accuracies (0.95). Sixteen percent of physicians performed below the competent care standard, while 22 % met the excellent care standard. Specialists scored higher than generalists, and better practice infrastructure was associated with higher composite scores, providing some validity evidence. CONCLUSIONS: We developed a rigorous methodology for assessing physicians' osteoporosis care. Clinical performance feedback relative to absolute standards of care provides physicians with a meaningful approach to self-evaluation to improve patient care.

Assessing interprofessional teamwork: pilot test of a new assessment module for practicing physicians.

INTRODUCTION: Teamwork is a basic component of all health care, and substantial research links the quality of teamwork to safety and quality of care. The TEAM (Teamwork Effectiveness Assessment Module) is a new Web-based teamwork assessment module for practicing hospital physicians. The module combines self-assessment, multisource feedback from members of other professions and specialties with whom the physician exercises teamwork, and a structured review of those data with a peer to develop an improvement plan. METHODS: We conducted a pilot test of this module with hospitalist physicians to evaluate the feasibility and usefulness of the module in practice, focusing on these specific questions: Would physicians in hospitals of different types and sizes be able to use the module; would the providers identified as raters respond to the request for feedback; would the physicians be able to identify one or more "trusted peers" to help analyze the feedback; and how would physicians experience the module process overall? RESULTS: 20 of 25 physicians who initially volunteered for the pilot completed all steps of the TEAM, including identifying interprofessional teammates, soliciting feedback from their team, and identifying a peer to help review data. Module users described the feedback they received as helpful and actionable, and indicated this was information they would not have otherwise received. CONCLUSIONS: The results suggest that a module combining self-assessment, multisource feedback, and a guided process for interpreting these data can provide help practicing hospital physicians to understand and potentially improve their interprofessional teamwork skills and behaviors.

Association between imposition of a Maintenance of Certification requirement and ambulatory care-sensitive hospitalizations and health care costs.

IMPORTANCE: In 1990, the American Board of Internal Medicine (ABIM) ended lifelong certification by initiating a 10-year Maintenance of Certification (MOC) program that first took effect in 2000. Despite the importance of this change, there has been limited research examining associations between the MOC requirement and patient outcomes. OBJECTIVE: To measure associations between the original ABIM MOC requirement and outcomes of care. DESIGN, SETTING, AND PARTICIPANTS: Quasi-experimental comparison between outcomes for Medicare beneficiaries treated in 2001 by 2 groups of ABIM-certified internal medicine physicians (general internists). One group (n = 956), initially certified in 1991, was required to fulfill the MOC program in 2001 (MOC-required) and treated 84 215 beneficiaries in the sample; the other group (n = 974), initially certified in 1989, was grandfathered out of the MOC requirement (MOC-grandfathered) and treated 69 830 similar beneficiaries in the sample. We compared differences in outcomes for the beneficiary cohort treated by the MOC-required general internists before (1999-2000) and after (2002-2005) they were required to complete MOC, using the beneficiary cohort treated by the MOC-grandfathered general internists as the control. MAIN OUTCOMES AND MEASURES: Quality measures were ambulatory care-sensitive hospitalizations (ACSHs), measured using prevention quality indicators. Ambulatory care-sensitive hospitalizations are hospitalizations triggered by conditions thought to be potentially preventable through better access to and quality of outpatient care. Other outcomes included health care cost measures (adjusted to 2013 dollars). RESULTS: Annual incidence of ACSHs (per 1000 beneficiaries) increased from the pre-MOC period (37.9 for MOC-required beneficiaries vs 37.0 for MOC-grandfathered beneficiaries) to the post-MOC period (61.8 for MOC-required beneficiaries vs 61.4 for MOC-grandfathered beneficiaries) for both cohorts, as did annual per-beneficiary health care costs (pre-MOC period, $5157 for MOC-required beneficiaries vs $5133 for MOC-grandfathered beneficiaries; post-MOC period, $7633 for MOC-required beneficiaries vs $7793 for MOC-grandfathered beneficiaries). The MOC requirement was not statistically associated with cohort differences in the growth of the annual ACSH rate (per 1000 beneficiaries, 0.1 [95% CI, -1.7 to 1.9]; P = .92), but was associated with a cohort difference in the annual, per-beneficiary cost growth of -$167 (95% CI, -$270.5 to -$63.5; P = .002; 2.5% of overall mean cost). CONCLUSION AND RELEVANCE: Imposition of the MOC requirement was not associated with a difference in the increase in ACSHs but was associated with a small reduction in the growth differences of costs for a cohort of Medicare beneficiaries.

Development of an instrument to evaluate residents' confidence in quality improvement.

BACKGROUND: Practice-based learning and improvement is a core competency that all medical residents must demonstrate. Because confidence is important in translating competence into action, effective quality improvement (QI) curricula should evaluate trainees' knowledge and confidence to perform QI. Past efforts to assess educational outcomes in QI have not adequately evaluated trainees' confidence from a multidimensional perspective. METHODS: Participants--732 internal medicine and family medicine residents from 42 training programs in the United States--completed the 31-item Quality Improvement Confidence Instrument (QICI), which was developed to measure confidence in six QI skill domains based on the Institute for Healthcare Improvement model ofQI. Confirmatory factor analysis was performed to support construct validity. Multivariate analysis of covariance was used to examine associations between residents' QI experience and other characteristics with confidence scores. RESULTS: Confirmatory factor analysis supported the QICI's multidimensional structure. Individual items yielded adequate variability, and reliability estimates for all six domains were high (> 0.86). On average, residents rated their confidence lowest for skills pertaining to choosing a target for improvement (specifically, using methods to evaluate interventions and to identify sources of process errors) and for testing a change in practice using specific tools for data collection and analysis. After controlling for program year and other characteristics, residents with previous QI experience reported significantly greater QI confidence. CONCLUSION: The QICI offers a psychometrically rigorous approach to evaluating residents' confidence levels. It can be used to gauge the appropriateness of a trainee's confidence against actual QI performance.

A new tool to give hospitalists feedback to improve interprofessional teamwork and advance patient care.

Teamwork is a vital skill for health care professionals, but the fragmented systems within which they work frequently do not recognize or support good teamwork. The American Board of Internal Medicine has developed and is testing the Teamwork Effectiveness Assessment Module (TEAM), a tool for physicians to evaluate how they perform as part of an interprofessional patient care team. The assessment provides hospitalist physicians with feedback data drawn from their own work of caring for patients, in a way that is intended to support immediate, concrete change efforts to improve the quality of patient care. Our approach demonstrates the value of looking at teamwork in the real world of health care-that is, as it occurs in the actual contexts in which providers work together to care for patients. The assessment of individual physicians' teamwork competencies may play a role in the larger effort to bring disparate health professions together in a system that supports and rewards a team approach in hope of improving patient care.

Comparative trial of a web-based tool to improve the quality of care provided to older adults in residency clinics: modest success and a tough road ahead.

PURPOSE: To determine whether residency programs can use a multicomponent, Web-based quality improvement tool to improve the care of older adults. METHOD: The authors conducted an exploratory, cluster-randomized, comparative before-after trial of the Care of the Vulnerable Elderly Practice Improvement Module in the ambulatory clinics of 46 internal medicine and family medicine residency programs, 2006-2008. The main outcomes were the deltas between pre- and post-performance on the Assessing Care of the Vulnerable Elderly (ACOVE) quality measures. RESULTS: Of the 46 programs initially selected for the study, 37 (80%) provided both baseline and follow-up data. Performance on all 10 ACOVE measures was poor at baseline (range 8.6%-33.6%). Intervention clinics most frequently chose for improvement fall-risk screening and documentation of end-of-life preferences. The change in the percentage of patients screened for fall risk for the intervention clinics that targeted this measure was significantly greater than the change observed by the control clinics (+23.3% versus +9.7%, P = .003, odds ratio [OR] = 2.0; 95% confidence interval [CI]: 1.25-3.75), as was the difference observed for documentation of preference for life-sustaining care (+16.4% versus +2.8%, P = .002, OR = 6.3; 95% CI: 2.0-19.6) and surrogate decision maker (+14.3% versus +2.8%, P = .003, OR = 6.8; 95% CI: 1.9-24.4). CONCLUSIONS: A multicomponent, Web-based, quality improvement tool can help residency programs improve care for older adults, but much work remains for improving the state of care for this population in training settings.

Gaps in quality of diabetes care in internal medicine residency clinics suggest the need for better ambulatory care training.

To ensure that medical residents will be prepared to deliver consistently high-quality care, they should be trained in settings that provide such care. Residents in internal medicine, particularly, need to learn good care habits in order to meet the needs of patients with diabetes and other common chronic and high-impact illnesses. To assess the strength of such training, we compared the quality of medical care provided in sixty-seven US internal medicine residency ambulatory clinics with the quality of care provided by 703 practicing general internists. We found significant quality gaps in process, intermediate outcome, and patient-experience measures. These inadequacies in ambulatory training for internal medicine residents must be addressed by policy makers and educators-for example, by accelerating the movement toward new residency curricula that emphasize competency-based training.

Variability in obtaining institutional review board approval for quality improvement activities in residency programs.

INTRODUCTION: Quality improvement (QI) activities are an important part of residency training. National studies are needed to inform best practices in QI training and experience for residents. The impact of the Institutional Review Board (IRB) process on such studies is not well described. METHODS: This observational study looked at time, length, comfort level, and overall quality of experience for 42 residency training programs in obtaining approval or exemption for a nationally based educational QI study. RESULTS: For the 42 programs in the study, the time period to IRB approval/exemption was highly variable, ranging from less than 1 week to 56.5 weeks; mean and median time was approximately 18 weeks (SD, 10.8). Greater reported comfort with the IRB process was associated with less time to obtain approval (r  =  -.50; P < .01; 95% CI, -0.70 to -0.23). A more positive overall quality of experience with the IRB process was also associated with less time to obtain IRB approval (r  =  -.60; P < .01; 95% CI, -0.74 to -0.36). DISCUSSION: The IRB process for residency programs initiating QI studies shows considerable variance that is not explained by attributes of the projects. New strategies are needed to assist and expedite IRB processes for QI research in educational settings and reduce interinstitutional variability and increase comfort level among educators with the IRB process.

Listening to older adults: elderly patients' experience of care in residency and practicing physician outpatient clinics.

The population of people aged 65 and older is rapidly growing. Research has demonstrated significant quality gaps in the clinical care of older patients in the United States, especially in training programs. Little is known about how older patients' experience with care delivered in residency clinics compares with that delivered by practicing physicians. Using patient surveys from the American Board of Internal Medicine Care of the Vulnerable Elderly Practice Improvement Module, the quality of care provided to adults aged 65 and older by 52 internal medicine and family medicine residency clinics and by a group of 144 practicing physicians was studied. The residency clinics received 2,213 patient surveys, and the practicing physicians received 4,204. Controlling for age and overall health status, patients from the residency clinic sample were less likely to report receiving guidance and interventions for important aspects of care for older adults than patients from the practicing physician sample. The largest difference was observed in providing ways to help patients prevent falls or treat problems with balance or walking (42.1% vs 61.8%, P<.001). Patients from the residency clinic sample were less likely to rate their overall care as high (77.5% vs 88.8%, P<.001). Patient surveys reveal important deficiencies in processes of care that are more pronounced for patients cared for in residency clinics. Quality of patient experience and communication are vital aspects of overall quality of care, especially for older adults. Physician education at all levels, faculty development, and practice system redesign are needed to ensure that the care needs of older adults are met.

Setting a fair performance standard for physicians' quality of patient care.

BACKGROUND: Assessing physicians' clinical performance using statistically sound, evidence-based measures is challenging. Little research has focused on methodological approaches to setting performance standards to which physicians are being held accountable. OBJECTIVE: Determine if a rigorous approach for setting an objective, credible standard of minimally-acceptable performance could be used for practicing physicians caring for diabetic patients. DESIGN: Retrospective cohort study. PARTICIPANTS: Nine hundred and fifty-seven physicians from the United States with time-limited certification in internal medicine or a subspecialty. MAIN MEASURES: The ABIM Diabetes Practice Improvement Module was used to collect data on ten clinical and two patient experience measures. A panel of eight internists/subspecialists representing essential perspectives of clinical practice applied an adaptation of the Angoff method to judge how physicians who provide minimally-acceptable care would perform on individual measures to establish performance thresholds. Panelists then rated each measure's relative importance and the Dunn-Rankin method was applied to establish scoring weights for the composite measure. Physician characteristics were used to support the standard-setting outcome. KEY RESULTS: Physicians abstracted 20,131 patient charts and 18,974 patient surveys were completed. The panel established reasonable performance thresholds and importance weights, yielding a standard of 48.51 (out of 100 possible points) on the composite measure with high classification accuracy (0.98). The 38 (4%) outlier physicians who did not meet the standard had lower ratings of overall clinical competence and professional behavior/attitude from former residency program directors (p = 0.01 and p = 0.006, respectively), lower Internal Medicine certification and maintenance of certification examination scores (p = 0.005 and p < 0.001, respectively), and primarily worked as solo practitioners (p = 0.02). CONCLUSIONS: The standard-setting method yielded a credible, defensible performance standard for diabetes care based on informed judgment that resulted in a reasonable, reproducible outcome. Our method represents one approach to identifying outlier physicians for intervention to protect patients.

Measuring physicians' performance in clinical practice: reliability, classification accuracy, and validity.

Much research has been devoted to addressing challenges in achieving reliable assessments of physicians' clinical performance but less work has focused on whether valid and accurate classification decisions are feasible. This study used 957 physicians certified in internal medicine (IM) or a subspecialty, who completed the American Board of Internal Medicine (ABIM) Diabetes Practice Improvement Module (PIM). Ten clinical and two patient-experience measures were aggregated into a composite measure. The composite measure score was highly reliable (r = .91) and classification accuracy was high across the entire score scale (>0.90), which indicated that it is possible to differentiate high-performing and low-performing physicians. Physicians certified in endocrinology and those who scored higher on their IM certification examination had higher composite scores, providing some validity evidence. In summary, it is feasible to create a psychometrically robust composite measure of physicians' clinical performance, specifically for the quality of care they provide to patients with diabetes.