Machine Learning to Identify Clusters in Family Medicine Diplomate Motivations and Their Relationship to Continuing Certification Exam Outcomes: Findings and Potential Future Implications.

BACKGROUND: The potential for machine learning (ML) to enhance the efficiency of medical specialty boards has not been explored. We applied unsupervised ML to identify archetypes among American Board of Family Medicine (ABFM) Diplomates regarding their practice characteristics and motivations for participating in continuing certification, then examined associations between motivation patterns and key recertification outcomes. METHODS: Diplomates responding to the 2017 to 2021 ABFM Family Medicine continuing certification examination surveys selected motivations for choosing to continue certification. We used Chi-squared tests to examine difference proportions of Diplomates failing their first recertification examination attempt who endorsed different motivations for maintaining certification. Unsupervised ML techniques were applied to generate clusters of physicians with similar practice characteristics and motivations for recertifying. Controlling for physician demographic variables, we used logistic regression to examine the effect of motivation clusters on recertification examination success and validated the ML clusters by comparison with a previously created classification schema developed by experts. RESULTS: ML clusters largely recapitulated the intrinsic/extrinsic framework devised by experts previously. However, the identified clusters achieved a more equal partitioning of Diplomates into homogenous groups. In both ML and human clusters, physicians with mainly extrinsic or mixed motivations had lower rates of examination failure than those who were intrinsically motivated. DISCUSSION: This study demonstrates the feasibility of using ML to supplement and enhance human interpretation of board certification data. We discuss implications of this demonstration study for the interaction between specialty boards and physician Diplomates.

Enhancing Patient Communication With Chat-GPT in Radiology: Evaluating the Efficacy and Readability of Answers to Common Imaging-Related Questions.

PURPOSE: To assess ChatGPT's accuracy, relevance, and readability in answering patients' common imaging-related questions and examine the effect of a simple prompt. METHODS: A total of 22 imaging-related questions were developed from categories previously described as important to patients, as follows: safety, the radiology report, the procedure, preparation before imaging, meaning of terms, and medical staff. These questions were posed to ChatGPT with and without a short prompt instructing the model to provide an accurate and easy-to-understand response for the average person. Four board-certified radiologists evaluated the answers for accuracy, consistency, and relevance. Two patient advocates also reviewed responses for their utility for patients. Readability was assessed using the Flesch Kincaid Grade Level. Statistical comparisons were performed using χ2 and paired t tests. RESULTS: A total of 264 answers were assessed for both unprompted and prompted questions. Unprompted responses were accurate 83% of the time (218 of 264), which did not significantly change for prompted responses (87% [229 of 264]; P = .2). The consistency of the responses increased from 72% (63 of 88) to 86% (76 of 88) when prompts were given (P = .02). Nearly all responses (99% [261 of 264]) were at least partially relevant for both question types. Fewer unprompted responses were considered fully relevant at 67% (176 of 264), although this increased significantly to 80% when prompts were given (210 of 264; P = .001). The average Flesch Kincaid Grade Level was high at 13.6 [CI, 12.9-14.2], unchanged with the prompt (13.0 [CI, 12.41-13.60], P = .2). None of the responses reached the eighth-grade readability level recommended for patient-facing materials. DISCUSSION: ChatGPT demonstrates the potential to respond accurately, consistently, and relevantly to patients' imaging-related questions. However, imperfect accuracy and high complexity necessitate oversight before implementation. Prompts reduced response variability and yielded more-targeted information, but they did not improve readability. ChatGPT has the potential to increase accessibility to health information and streamline the production of patient-facing educational materials; however, its current limitations require cautious implementation and further research.

Evidence for an adverse impact of remote readouts on radiology resident productivity: Implications for training and clinical practice.

After their rapid adoption at the onset of the coronavirus pandemic, remote case reviews (remote readouts) between diagnostic radiology residents and their attendings have persisted in an increasingly remote workforce, despite relaxing social distancing guidelines. Our objective was to evaluate the impact of the transition to remote readouts on resident case volumes after the recovery of institutional volumes. We tabulated radiology reports co-authored by first-to-third-year radiology residents (R1-R3) between July 1 and December 31 of the first pandemic year, 2020, and compared to the prior two pre-pandemic years. Half-years were analyzed because institutional volumes recovered by July 2020. Resident volumes were normalized to rotations, which were in divisions categorized by the location of the supervising faculty during the pandemic period; in 'remote' divisions, all faculty worked off-site, whereas 'hybrid' divisions had a mix of attendings working on-site and remotely. All residents worked on-site. Data analysis was performed with Student's t test and multivariate linear regression. The largest drops in total case volume occurred in the two remote divisions (38% [6,086 to 3,788], and 26% [11,046 to 8,149]). None of the hybrid divisions with both in-person and remote supervision decreased by more than 5%. With multivariate regression, a resident assigned to a standardized remote rotation in 2020 would complete 32% (253 to 172) fewer studies than in identical pre-pandemic rotations (coefficent of -81.6, p = .005) but would be similar for hybrid rotations. R1 residents would be expected to interpret 40% fewer (180 to 108) cases on remote rotations during the pandemic (coefficient of -72.3, p = .007). No significant effect was seen for R2 or R3 residents (p = .099 and p = .29, respectively). Radiology residents interpreted fewer studies during remote rotations than on hybrid rotations that included in-person readouts. As resident case volume is correlated with clinical performance and board pass rate, monitoring the readout model for downstream educational effects is essential. Until evidence shows that educational outcomes remain unchanged, radiology residencies may wish to preserve in-person resident readouts, particularly for junior residents.

A Novel Approach to Teaching Fundoscopy Using a Virtual Format.

Introduction: Ophthalmology education has been underemphasized in medical school curricula despite the fact that patient eye-related complaints are commonplace across primary care specialties. Although previous curricula used direct ophthalmoscopy to teach medical students the fundamentals of ophthalmic examination, there has been a growing call to teach these fundamentals through reading fundus photos due to the increasing prevalence and decreased costs of fundus cameras in primary care settings. We developed a virtual workshop to teach ophthalmoscopy to medical students using fundus photography. Methods: First-year medical students were enrolled in a 2-hour, synchronous, virtual ophthalmoscopy workshop as part of an advanced physical exam curriculum at the University of Pittsburgh School of Medicine. Students participated in a pretest, introductory lecture, interactive small-group session, and posttest. Breakout groups were led by senior medical students or residents. We compared pre- and posttest results for improved understanding of concepts covered in the workshop. Results: Of 147 students, the average scores on the pretest and posttest were 39% and 75%, respectively (p < .01). Students were significantly more confident in their ability to identify various pathologies on fundus photography. After the workshop, the student preceptors indicated increased comfort in a teaching role and greater interest in medical education. The preceptors were also more confident in their own ability to interpret fundus photography and in their understanding of various ocular pathologies. Discussion: Our virtual, interactive workshop is effective in teaching medical students a systematic approach to the interpretation of fundus photographs.

Using Machine Learning to Predict Primary Care and Advance Workforce Research.

PURPOSE: To develop and test a machine-learning-based model to predict primary care and other specialties using Medicare claims data. METHODS: We used 2014-2016 prescription and procedure Medicare data to train 3 sets of random forest classifiers (prescription only, procedure only, and combined) to predict specialty. Self-reported specialties were condensed to 27 categories. Physicians were assigned to testing and training cohorts, and random forest models were trained and then applied to 2014-2016 data sets for the testing cohort to generate a series of specialty predictions. Comparing the predicted specialty to self-report, we assessed performance with F1 scores and area under the receiver operating characteristic curve (AUROC) values. RESULTS: A total of 564,986 physicians were included. The combined model had a greater aggregate (macro) F1 score (0.876) than the prescription-only (0.745; P <.01) or procedure-only (0.821; P <.01) model. Mean F1 scores across specialties in the combined model ranged from 0.533 to 0.987. The mean F1 score was 0.920 for primary care. The mean AUROC value for the combined model was 0.992, with values ranging from 0.982 to 0.999. The AUROC value for primary care was 0.982. CONCLUSIONS: This novel approach showed high performance and provides a near real-time assessment of current primary care practice. These findings have important implications for primary care workforce research in the absence of accurate data.

Primary Care Physician Characteristics Associated with Low Value Care Spending.

BACKGROUND: Previous work has shown that $210 billion may be spent annually on unnecessary medical services and has identified patient and hospital characteristics associated with low value care (LVC). However, little is known about the association between primary care physician (PCP) characteristics and LVC spending. The objective of this study was to assess this association. METHODS: We performed a retrospective analysis by using Medicare claims data to identify LVC and American Medical Association Masterfile data for PCP characteristics. We included PCPs of adults aged 65 years and older who were enrolled in Medicare in 2011. We measured Medicare spending per attributed patient on 8 low value services. RESULTS: Our final sample contained 6,873 PCPs with 1,078,840 attributed patients. Lower per-patient LVC Medicare spending was associated with the following PCP characteristics: allopathic training, smaller Medicare patient panel, practiced family medicine, practiced in the Midwest region, were a recent graduate, or practiced in rural areas. The largest associations were seen in Medicare patient panel size and geographic region. The average per-patient LVC spending was $14.67. LVC spending among PCPs with small patient panels was $3.98 less per patient relative to those with larger panels. PCPs in the Midwest had $2.80 less per patient LVC spending than those in the Northeast. CONCLUSION: Our analysis suggests that LVC services are associated with specific PCP characteristics. Further research should assess the strength of these associations, and future policy efforts should focus on systemic interventions to reduce LVC spending.

Characteristics of Young Family Physician Hospitalists.

Based on a 2016 survey of family physicians who were then three years out of residency training, we found that almost 9 percent self-identified as hospitalists. These family physician hospitalists were significantly more likely than their non-hospitalist peers to be male, work longer hours, be better paid, and be more satisfied with their work. These attributes may attract more family physicians to hospital medicine, with negative implications for the supply of primary care physicians. (J Am Board Fam Med 2018;31:680-681.).

The Growing Trend of Health Coaches in Team-Based Primary Care Training A Multicenter Pilot Study.

BACKGROUND AND OBJECTIVES: Team-based care with health coaches has improved the quality and cost effectiveness of chronic disease management and prevention. Clinical health coaches partner with patients to identify health goals, create action plans, overcome barriers to change, reinforce physician recommendations, and coordinate care. It is important to train resident physicians to practice in team-based settings. To date, there have been no studies of resident family physician exposure to health coaches. METHODS: We surveyed 465 residency directors through a larger omnibus survey sent out by CERA; the response rate was 53.7%. Directors were asked about resident exposure to health coaches, the types of patients seen by health coaches, and the training background of the health coaches. We used chi-square tests to examine the relationship among these variables and program characteristics including status as a patient-centered medical home. RESULTS: Almost two-thirds of the programs reported at least some residents had exposure to health coaches. Residents who trained in continuity sites with a PCMH certification of level 3 were more likely to have any exposure to health coaches (P<.05). There were multiple significant relationships between populations of patients seen and the training background of health coaches. CONCLUSIONS: To improve quality, reduce costs, and become more patient centered, primary care is rapidly transforming into a team sport with a broadening roster of new players, including health coaches. This study documents positive rates of resident exposure to health coaches but also great variability in types and amount of exposure that merit further investigation and exploration of ways to grow family medicine residency contact with a diversifying practice team.

Predictors of Attrition From Family Medicine Board Certification.

Board certification is associated with higher quality care. We sought to determine the rates and predictors of attrition from certification among family physicians who achieved initial certification with the American Board of Family Medicine from 1980 through 2000. In this period, 5.6% of family physicians never attempted recertification, with the rate increasing from 4.9% between 1990 and 1995 to 5.7% from 1996 to 2000. Being male, an international medical graduate, or 30 years of age or older at initial certification was associated with not recertifying. With information about those likely to leave certification, the board can design and implement interventions that minimize attrition.

Using Drug Prescribing Patterns to Identify Stewards of Cost-Conscious Care.

PURPOSE: To characterize family physicians (FPs) who are stewards of care by consistently prescribing omeprazole over esomeprazole. METHODS: Cross-sectional analysis of physicians prescribing omeprazole or esomeprazole under Medicare Part D in 2014. RESULTS: There was a regional trend with 49% of Western FPs but only 6% of Southern FPs rarely prescribing esomeprazole. Physicians had increased odds of being a steward if they worked with a care coordinator (P < .001), at a patient-centered medical home (P < .001), or in a large practice (P < .001). CONCLUSIONS: If these findings are replicated across multiple drugs, future outreach could be conducted based on provider prescribing patterns.

Mobility of US Rural Primary Care Physicians During 2000-2014.

PURPOSE: Despite considerable investment in increasing the number of primary care physicians in rural shortage areas, little is known about their movement rates and factors influencing their mobility. We aimed to characterize geographic mobility among rural primary care physicians, and to identify location and individual factors that influence such mobility. METHODS: Using data from the American Medical Association Physician Masterfile for each clinically active US physician, we created seven 2-year (biennial) mobility periods during 2000-2014. These periods were merged with county-level "rurality," physician supply, economic characteristics, key demographic measures, and individual physician characteristics. We computed (1) mobility rates of physicians by rurality; (2) linear regression models of county-level rural nonretention (departure); and (3) logit models of physicians leaving rural practice. RESULTS: Biennial turnover was about 17% among physicians aged 45 and younger, compared with 9% among physicians aged 46 to 65, with little difference between rural and metropolitan groups. County-level physician mobility was higher for counties that lacked a hospital (absolute increase = 5.7%), had a smaller population size, and had lower primary care physician supply, but area-level economic and demographic factors had little impact. Female physicians (odds ratios = 1.24 and 1.46 for those aged 45 or younger and those aged 46 to 65, respectively) and physicians born in a metropolitan area (odds ratios = 1.75 and 1.56 for those aged 45 or younger and those aged 46 to 65, respectively) were more likely to leave rural practice. CONCLUSIONS: These flndings provide national-level evidence of rural physician mobility rates and factors associated with both county-level retention and individual-level departures. Outcomes were notably poorer in the most remote locations and those already having poorer physician supply and professional support. Rural health workforce planners and policymakers must be cognizant of these key factors to more effectively target retention policies and to take into account the additional support needed by these more vulnerable communities.

Funding Instability Reduces the Impact of the Federal Teaching Health Center Graduate Medical Education Program.

The Teaching Health Center Graduate Medical Education (THCGME) program is a decentralized residency training component of the Affordable Care Act, created to combat critical shortages and maldistribution of primary care physicians. The Accreditation Council of Graduate Medical Education and federal data reveal that the THCGME program accounted for 33% of the net increase in family medicine residency positions between 2011 and 2015. However, amid concerns about the program's stability, the contribution of the THCGME program to the net increase fell to 7% after 2015.

Measuring the attractiveness of rural communities in accounting for differences of rural primary care workforce supply.

INTRODUCTION: Many rural communities continue to experience an undersupply of primary care doctor services. While key professional factors relating to difficulties of recruitment and retention of rural primary care doctors are widely identified, less attention has been given to the role of community and place aspects on supply. Place-related attributes contribute to a community's overall amenity or attractiveness, which arguably influence both rural recruitment and retention relocation decisions of doctors. This bi-national study of Australia and the USA, two developed nations with similar geographic and rural access profiles, investigates the extent to which variations in community amenity indicators are associated with spatial variations in the supply of rural primary care doctors. METHODS: Measures from two dimensions of community amenity: geographic location, specifically isolation/proximity; and economics and sociodemographics were included in this study, along with a proxy measure (jurisdiction) of a third dimension, environmental amenity. Data were chiefly collated from the American Community Survey and the Australian Census of Population and Housing, with additional calculated proximity measures. Rural primary care supply was measured using provider-to-population ratios in 1949 US rural counties and in 370 Australian rural local government areas. Additionally, the more sophisticated two-step floating catchment area method was used to measure Australian rural primary care supply in 1116 rural towns, with population sizes ranging from 500 to 50 000. Associations between supply and community amenity indicators were examined using Pearson's correlation coefficients and ordinary least squares multiple linear regression models. RESULTS: It was found that increased population size, having a hospital in the county, increased house prices and affluence, and a more educated and older population were all significantly associated with increased workforce supply across rural areas of both countries. While remote areas were strongly linked with poorer supply in Australia, geographical remoteness was not significant after accounting for other indicators of amenity such as the positive association between workforce supply and coastal location. Workforce supply in the USA was negatively associated with fringe rural area locations adjacent to larger metropolitan areas and characterised by long work commutes. The US model captured 49% of the variation of workforce supply between rural counties, while the Australian models captured 35-39% of rural supply variation. CONCLUSIONS: These data support the idea that the rural medical workforce is maldistributed with a skew towards locating in more affluent and educated areas, and against locating in smaller, poorer and more isolated rural towns, which struggle to attract an adequate supply of primary care services. This evidence is important in understanding the role of place characteristics and rural population dynamics in the recruitment and retention of rural doctors. Future primary care workforce policies need to place a greater focus on rural communities that, for a variety of reasons, may be less attractive to doctors looking to begin or remain working there.

Regional Variation in Primary Care Involvement at the End of Life.

PURPOSE: Variation in end-of-life care in the United States is frequently driven by the health care system. We assessed the association of primary care physician involvement at the end of life with end-of-life care patterns. METHODS: We analyzed 2010 Medicare Part B claims data for US hospital referral regions (HRRs). The independent variable was the ratio of primary care physicians to specialist visits in the last 6 months of life. Dependent variables included the rate of hospital deaths, hospital and intensive care use in the last 6 months of life, percentage of patients seen by more than 10 physicians, and Medicare spending in the last 2 years of life. Robust linear regression analysis was used to measure the association of primary care physician involvement at the end of life with the outcome variables, adjusting for regional characteristics. RESULTS: We assessed 306 HRRs, capturing 1,107,702 Medicare Part B beneficiaries with chronic disease who died. The interquartile range of the HRR ratio of primary care to specialist end-of-life visits was 0.77 to 1.21. HRRs with high vs low primary care physician involvement at the end of life had significantly different patient, population, and health system characteristics. Adjusting for these differences, HRRs with the greatest primary care physician involvement had lower Medicare spending in the last 2 years of life ($65,160 vs $69,030; P = .003) and fewer intensive care unit days in the last 6 months of life (2.90 vs 4.29; P <.001), but also less hospice enrollment (44.5% of decedents vs 50.4%; P = .004). CONCLUSIONS: Regions with greater primary care physician involvement in end-of-life care have overall less intensive end-of-life care.

The Effects of Training Institution Practice Costs, Quality, and Other Characteristics on Future Practice.

PURPOSE: Medicare beneficiary spending patterns reflect those of the 306 Hospital Referral Regions where physicians train, but whether this holds true for smaller areas or for quality is uncertain. This study assesses whether cost and quality imprinting can be detected within the 3,436 Hospital Service Areas (HSAs), 82.4 percent of which have only 1 teaching hospital, and whether sponsoring institution characteristics are associated. METHODS: We conducted a secondary, multi-level, multivariable analysis of 2011 Medicare claims and American Medical Association Masterfile data for a random, nationally representative sample of family physicians and general internists who completed residency between 1992 and 2010 and had more than 40 Medicare patients (3,075 physicians providing care to 503,109 beneficiaries). Practice and training locations were matched with Dartmouth Atlas HSAs and categorized into low-, average-, and high-cost spending groups. Practice and training HSAs were assessed for differences in 4 diabetes quality measures. Institutional characteristics included training volume and percentage of graduates in rural practice and primary care. RESULTS: The unadjusted, annual, per-beneficiary spending difference between physicians trained in high- and low-cost HSAs was $1,644 (95% CI, $1,253-$2,034), and the difference remained significant after controlling for patient and physician characteristics. No significant relationship was found for diabetes quality measures. General internists were significantly more likely than family physicians to train in high-cost HSAs. Institutions with more graduates in rural practice and primary care produced lower-spending physicians. CONCLUSIONS: The "imprint" of training spending patterns on physicians is strong and enduring, without discernible quality effects, and, along with identified institutional features, supports measures and policy options for improved graduate medical education outcomes.

Changes in Primary Care Graduate Medical Education Are Not Correlated With Indicators of Need: Are States Missing an Opportunity to Strengthen Their Primary Care Workforce?

PURPOSE: Federal and state graduate medical education (GME) funding exceeds $15 billion annually. It is critical to understand mechanisms to align undergraduate medical education (UME) and GME to meet workforce needs. This study aimed to determine whether states' primary care GME (PCGME) trainee growth correlates with indicators of need. METHOD: Data from the American Medical Association Physician Masterfile, the Association of American Medical Colleges, the American Association of the Colleges of Osteopathic Medicine, and the U.S. Census were analyzed to determine how changes between 2002 and 2012 in PCGME trainees-a net primary care physician (PCP) production estimate-correlated with state need using three indicators: (1) PCP-to-population ratio, (2) change in UME graduates, and (3) population growth. RESULTS: Nationally, PCGME trainees declined by 7.1% from the net loss of 679 trainees (combined loss of 54 postgraduate year 1 trainees in internal medicine, family medicine, and pediatrics and addition of 625 fellowship trainees in those specialties). The median state PCGME decline was 2.7%. There was no correlation between the percent change in states' PCGME trainees and PCP-to-population ratio (r = -0.06) or change in UME graduates (r = 0.17). Once adjusted for population growth, PCGME trainees declined by 15.3% nationally; the median state decline was 9.7%. CONCLUSIONS: There is little relationship between PCGME trainee growth and state need indicators. States should capitalize on opportunities to create explicit linkages between UME, GME, and population need; strategically allocate Medicaid GME funds; and monitor the impact of workforce policies and training institution outputs.

You can't treat what you don't diagnose: An analysis of the recognition of somatic presentations of depression and anxiety in primary care.

BACKGROUND: Research suggests that 13-25% of primary care patients who present with physical complaints have underlying depression or anxiety. OBJECTIVE: The goal of this paper is to quantify and compare the frequency of the diagnosis of depression and anxiety in patients with a somatic reason for visit among primary care physicians across disciplines. METHOD: Data obtained from the National Ambulatory Medical Care Survey (NAMCS) from 2002 to 2010 was used to quantify primary care patients with somatic presentations who were given a diagnosis of depression or anxiety. The Patient Health Questionnaire (PHQ)-15, Somatic Symptom Scale, and the Child Behavior Checklist for Ages 6-18 were used to define what constituted a somatic reason for visit in this study. RESULTS: Of the patients presenting with a somatic reason for visit in this nationally representative survey, less than 4% of patents in family or internal medicine were diagnosed with depression or anxiety. Less than 1% of patients were diagnosed with depression or anxiety in pediatrics or obstetrics and gynecology. Less than 2% of patients with somatic reasons for visit in any primary care specialty had documented screening for depression. CONCLUSION: The rates of diagnosis of depression and anxiety in patents presenting with somatic reasons for visit were significantly less than the prevalence reported in the literature across primary care disciplines. (PsycINFO Database Record