Performance of ChatGPT and Bard in self-assessment questions for nephrology board renewal.

BACKGROUND: Large language models (LLMs) have impacted advances in artificial intelligence. While LLMs have demonstrated high performance in general medical examinations, their performance in specialized areas such as nephrology is unclear. This study aimed to evaluate ChatGPT and Bard in their potential nephrology applications. METHODS: Ninety-nine questions from the Self-Assessment Questions for Nephrology Board Renewal from 2018 to 2022 were presented to two versions of ChatGPT (GPT-3.5 and GPT-4) and Bard. We calculated the correct answer rates for the five years, each year, and question categories and checked whether they exceeded the pass criterion. The correct answer rates were compared with those of the nephrology residents. RESULTS: The overall correct answer rates for GPT-3.5, GPT-4, and Bard were 31.3% (31/99), 54.5% (54/99), and 32.3% (32/99), respectively, thus GPT-4 significantly outperformed GPT-3.5 (p < 0.01) and Bard (p < 0.01). GPT-4 passed in three years, barely meeting the minimum threshold in two. GPT-4 demonstrated significantly higher performance in problem-solving, clinical, and non-image questions than GPT-3.5 and Bard. GPT-4's performance was between third- and fourth-year nephrology residents. CONCLUSIONS: GPT-4 outperformed GPT-3.5 and Bard and met the Nephrology Board renewal standards in specific years, albeit marginally. These results highlight LLMs' potential and limitations in nephrology. As LLMs advance, nephrologists should understand their performance for future applications.

Performance on the American Board of Physical Medicine and Rehabilitation certifying examinations: Rural and urban physicians.

BACKGROUND: Over 60 million people in the United States live in a rural community making up approximately 20% of the population. Data are minimal about the physiatrists who serve this rural population, their performance on certification examinations and how the American Board of Physical Medicine and Rehabilitation (ABPM&R) serves their ongoing educational, assessment, and practice needs. OBJECTIVE: To compare the performance of rural and urban physicians on the Part I, Part II, and maintenance of certification (MOC) examinations along with subspecialty preference and continuance of primary certification. DESIGN: Retrospective cross-sectional study. SETTING: Board-eligible PM&R physicians and certified diplomates of the ABPM&R. PARTICIPANTS: Physicians who participated in an initial certification or maintenance of certification examination with the ABPM&R between 2010 and 2019. METHODS: Comparisons of physician pass rates, mean scaled scores (aggregates), and program pass rates on ABPM&R certifying examinations were completed. Cross-reference to national database and ABPM&R practice site zip codes provided sociogeographic linkage. INTERVENTIONS: Not applicable MAIN OUTCOME MEASURES: Physician mean scaled scores, pass rates, subspecialty preferences, and primary certification status. RESULTS: There were no meaningful differences in performance on the ABPM&R Part I, II, and MOC examinations between rural and urban physiatrists. Most common subspecialty is the pain medicine certification whose diplomates most frequently drop their primary certification. Pediatric rehabilitation medicine certification is rare in rural localities and a health care disparity. CONCLUSION: The study found no meaningful differences in the performance of rural and urban physicians on the ABPM&R certifying examinations.

Fellowship Board Pass Rates Rising: Analysis of Pathology Subspecialty Board Examination Performance.

CONTEXT.­: The American Board of Pathology (ABPath) publishes annual performance data for the anatomic pathology (AP) and clinical pathology (CP) board examinations, as well as for ABPath subspecialty examinations. Overall board pass rates for all AP and CP board examinees have increased during the past decade; however, no study has analyzed the board pass rates for pathology subspecialty examinations, and whether these follow the same trend. OBJECTIVE.­: To evaluate ABPath subspecialty examination pass rates to assess the trend in certification. DESIGN.­: We analyzed the total number of first-time test takers and board pass rates for 11 pathology subspecialties recognized by the ABPath from 2007 to 2021, acquired from annual reports published by the ABPath. We compared the pass rates in 5-year intervals (2007-2011, 2012-2016, 2017-2021) for each individual specialty. We also analyzed the pass rate of CP subspecialties compared with AP subspecialties. RESULTS.­: The overall mean pass rate for ABPath subspecialty examinations during the previous 15 years was 89% (range, 78.9%-100%), with the overall pass rate being significantly higher in 2017-2021 (P = .02). The contemporary overall rate of passing was significantly higher for AP subspecialty examinations (P < .001) and was higher, though not significantly, for CP subspecialties (P = .13). There were significant differences between first-time test takers' mean pass rate (92.1%), repeat test takers' mean pass rate (54.5%), and the overall rate (P < .001). CONCLUSIONS.­: Contemporary pathology subspecialty board examination pass rates are significantly higher than historic rates, possibly reflecting continuously improving and readily available preparatory materials.

20 anos da comissão consultiva em avaliação psicológica: desafios da ADI 3481 / 20 years of the advisory committee on psychological assessment: challenges of ADI 3481 / 20 años de la comisión consultiva en evaluación psicológica: desafíos de la ADI 3481

A Comissão Consultiva em Avaliação Psicológica (CCAP) do Conselho Federal de Psicologia (CFP), em seu 20º aniversário, vem discutir os possíveis efeitos, ainda efetivamente desconhecidos, da Ação Direta de Inconstitucionalidade (ADI) 3481, instruída no Supremo Tribunal Federal (STF), a qual desestruturou o modo como os testes psicológicos eram comercializados no Brasil. A livre comercialização de testes psicológicos coloca em risco a segurança de avaliações psicológicas e cabe à categoria profissional pensar estratégias de enfrentamento desses riscos. Neste artigo, são discutidos possíveis efeitos da ADI 3481 para a categoria profissional da psicologia, bem como para a sociedade em geral, e são também elencadas possíveis estratégias de enfrentamento desses riscos, sem desconsiderar aspectos éticos relacionados a eles. Dessa forma, busca-se neste manuscrito, além da problematização dos efeitos derivados da ADI 3481, pensar soluções ou alternativas que venham a redirecionar a trajetória da área da avaliação psicológica no Brasil. Com isso, abre-se um espaço de discussão e encaminhamentos que a categoria profissional precisará tomar nos próximos anos.(AU)

The Advisory Commission for Psychological Assessment of the Federal Council of Psychology discusses, on its 20th anniversary, the possible and still effectively unknown effects of the Direct Action of Unconstitutionality (DAU) 3481, following the Supreme Federal Court, which interrupted how psychological tests were marketed in Brazil. The free trade of psychological tests puts the safety of psychological assessments at risk, and this professional category must think of strategies to face these risks. This study discusses the possible effects of DAU 3481 for professional psychology and for society in general, listing possible strategies for coping with these risks without disregarding its ethical aspects. Thus, this study seeks to problematize the effects derived from DAU 3481 and think of solutions or alternatives that may redirect the trajectory of the field of psychological assessment in Brazil, thus opening a space for discussion and referrals professional psychology will require in the coming years.(AU)

La Comisión Consultiva en Evaluación Psicológica (CCEP) del Consejo Federal de Psicología (CFP), en su 20.º aniversario, propone discutir los posibles efectos aún efectivamente desconocidos de la Acción Directa de Inconstitucionalidad (ADI) 3481, determinada por el Supremo Tribunal Federal (STF), por la cual trastornó la forma de comercializar las pruebas psicológicas en Brasil. La comercialización sin restricciones de las pruebas psicológicas pone en riesgo la seguridad de las evaluaciones psicológicas, y le corresponde a la categoría profesional pensar estrategias para enfrentar estos riesgos. En este artículo se discuten los posibles efectos de la ADI 3481 para la categoría profesional de la Psicología, así como para la sociedad en general, pero también se enumeran posibles estrategias para el enfrentamiento de estos riesgos, sin descuidar los aspectos éticos relacionados con ellos. Así, este manuscrito busca, además de problematizar los efectos derivados de la ADI 3481, pensar en soluciones o alternativas que puedan reconducir la trayectoria del campo de la evaluación psicológica en Brasil. Esto abre un espacio de discusión y derivaciones que la categoría profesional deberá tomar en los próximos años.(AU)

Board Certification in Cosmetic Surgery: An Analysis of Punitive Actions.

BACKGROUND: American Board of Plastic Surgery (ABPS) diplomates complete training in aesthetic surgery through an Accreditation Council of Graduate Medical Education-accredited program. American Board of Cosmetic Surgery (ABCS) diplomates complete residency training in a "related" specialty, some historically nonsurgical, followed by an American Association of Cosmetic Surgery fellowship. Unlike the ABPS, the ABCS is not recognized by the American Board of Medical Specialties as an equivalent certifying board. This study evaluated differences in the rates of punitive action against diplomates of the ABPS and the ABCS. METHODS: Diplomates were accessed from their respective society's websites. Punitive action data were obtained by search of publicly available state medical board databases. A comparative analysis was performed between ABPS and ABCS. RESULTS: One thousand two hundred eight physicians were identified for comparative analysis. Two hundred sixty-six (22 percent) were members of the American Society of Plastic Surgeons, and 549 (49 percent) were members of The Aesthetic Society. ABCS diplomates had significantly higher rates of disciplinary administrative action by their respective state medical boards [ n = 31 (9.0 percent)] when compared with ABPS members [The Aesthetic Society, n = 26 (4.4 percent); ABPS, n = 8 (3.1 percent); p = 0.003], with a higher proportion of repeat offenders. In addition, ABCS diplomates had more public letters of reprimand [ABCS, n = 12 (3.5 percent); The Aesthetic Society, n = 6 (1.2 percent); and ABPS, n = 2 (0.8 percent); p = 0.015]. CONCLUSIONS: ABCS diplomates have significantly higher rates of punitive actions than ABPS diplomates. Although the reasons for this discrepancy warrant further investigation, punitive data should be transparently and publicly available to aid patients in informed decision-making.

American Board of Physical Medicine and Rehabilitation Diplomate Customization Choices on the Longitudinal Assessment for Physical Medicine and Rehabilitation: A First-Year Experience.

ABSTRACT: The American Board of Physical Medicine and Rehabilitation began administering the longitudinal assessment for physical medicine and rehabilitation for continuing certification in 2020. The longitudinal assessment for physical medicine and rehabilitation digitally delivers quarterly short assessments of content and repeats missed items to facilitate learning as well as serve as a summative certification assessment. With a goal of offering content relevant to an individual's practice and learning needs, diplomates choose how to customize the domains or topic areas of their question content on an annual basis. This report describes the first year of experience with customization of longitudinal assessment for physical medicine and rehabilitation. The American Board of Physical Medicine and Rehabilitation diplomate customization data are grouped and compared in a variety of ways to ascertain whether there are differences in customization choices. While customization choices were similar across several domains, significant differences were seen when comparing groups with specific areas of practice or subspecialty certification. Smaller differences were also seen when comparing question domain allocation choice between sexes, age groups, and practice setting. The results from this first full year of experience confirm an alignment of this innovative assessment approach to individual physician practice, a significant step in improving the relevance of continuing certification overall for participating physicians.

Interrater Reliability in the American Board of Physical Medicine and Rehabilitation Part II Certification Examination: Impact of a New Assessment Design.

OBJECTIVE: The design of medical board certification examinations continues to evolve with advances in testing innovations and psychometric analysis. The potential for subjectivity is inherent in the design of oral board examinations, making improvements in reliability and validity especially important. The purpose of this quality improvement study was to analyze the impact of using two examiners on the overall reliability of the oral certification examination in physical medicine and rehabilitation. DESIGN: This was a retrospective quality improvement study of 422 candidates for the American Board of Physical Medicine and Rehabilitation Part II Examination in 2020. Candidates were examined by examiner pairs, each of whom submitted independent scores. Training for all 116 examiners included examination case review, scoring guidelines, and bias mitigation. Examiner performance was analyzed for both internal consistency (intrarater reliability) and agreement with their paired examiner (interrater reliability). RESULTS: The reliability of the Part II Examination was high, ranging from 0.93 to 0.94 over three administrations. The analysis also demonstrated high interrater agreement and examiner internal consistency. CONCLUSIONS: A high degree of interrater agreement was found using a new, two-examiner format. Comprehensive examiner training is likely the most significant factor for this finding. The two-examiner format improved the overall reliability and validity of the Part II Examination.

Disputas em torno da regulamentação da profissão: a psicologia em defesa das orientações sexuais e identidades de gênero / Disputes around the regulation of the profession: psychology in defense of sexual orientations and gender identities / Disputas sobre la regulación de la profesión: la psicología en defensa de las orientaciones sexuales e identidades de género

A normativa do Conselho Federal de Psicologia nº 1/1999, que estabelece normas de atuação para as(os) psicólogas(os) em relação à questão da orientação sexual, ao longo das últimas duas décadas tem sido objeto de investidas judiciais e legislativas para sustar seus efeitos. Foi observado também que a normativa CFP nº 01/2018, que se refere à atuação com pessoas transexuais e travestis, foi a mais rapidamente atacada após sua publicação, quando comparada aos mais de 1.500 atos oficiais publicados pelo Conselho Federal de Psicologia ao longo de sua história de quase 50 anos. E, mais recentemente, foi publicada uma normativa que trata da conduta profissional quanto às bissexualidades e demais orientações não monossexuais, a CFP nº08/2022. Este artigo tem como objetivo analisar os principais motivos das disputas e as estratégias construídas ao longo do processo de resistência pela Psicologia brasileira, em meio a tensões entre seu posicionamento, expresso pelas resoluções, e a sociedade.(AU)

Normative no. 01/99 of the Federal Council of Psychology, which establishes standards of action for psychologists regarding the issue of sexual orientation, over the last two decades has been the object of legal and legislative attacks to suppress its effects. Moreover, the CFP normative no. 01/18, which refers to working with transgender individuals and travestis, was the most quickly attacked, when compared to the more than 1,500 official acts published by the Federal Council of Psychology throughout its history of almost 50 years. More recently, a normative law was published on the professional conduct regarding bisexualities and other non-monosexual orientations-CFP no. 08/22. This article analyses the main reasons for the disputes and strategies built along the resistance process by Brazilian Psychology, amidst tensions between its positions, expressed by the resolutions, and society.(AU)

El reglamento del Consejo Federal de Psicología 01/99 que establece estándares de acción para psicólogas/os en relación con relación al tema de la orientación sexual, durante las últimas dos décadas, ha sido objeto de ataques judiciales y legislativos para reprimir sus efectos. Además, se observó que el reglamento del CPF 01/18 que trata la práctica con las personas transexuales y travestis fue el más rápidamente atacado, entre los más de 1.500 actos oficiales publicados por el Consejo Federal de Psicología a lo largo de su historia de casi 50 años. Y, más recientemente, se publicó el reglamento, el CPF 08/22, que plantea la acción profesional en relación con las bisexualidades y otras orientaciones no monosexuales. Este artículo tiene como objetivo analizar las principales razones de disputas y las estrategias construidas por la Psicología Brasileña a lo largo del proceso de resistencia en medio de las disputas entre su postura planteada mediante las resoluciones y la sociedad.(AU)

A Call for LGBTQ Content in Graduate Medical Education Program Requirements.

A well-developed body of literature demonstrates that lesbian, gay, bisexual, transgender, and queer (LGBTQ) individuals experience poorer health outcomes and report worse health care experiences than straight/cisgender individuals. Many reforms since 2010 have addressed the LGBTQ-related education of future health care professionals at the undergraduate medical education (UME) level; however, reforms at the graduate medical education (GME) level are lagging, and new literature suggests that didactic education at the UME level is not enough to prepare future physicians to properly and compassionately care for LGBTQ patients. Recently, the Accreditation Council for Graduate Medical Education (ACGME) implemented a major revision of its Common Program Requirements that requires residents to demonstrate, as a competence, respect and responsiveness to diverse populations. Given these revisions and the ongoing failure of many GME training programs to adequately prepare future physicians to care for LGBTQ patients, the authors argue that now is the time for the ACGME to develop and implement LGBTQ health-related residency requirements. In addition, the authors outline a path by which the academic medical community may develop and implement these requirements.

Coverage of obesity and obesity disparities on American Board of Medical Specialties (ABMS) examinations.

Obesity is a widespread disease which adversely impacts all organ systems and disproportionately affects African Americans and other minority groups. Physicians across medical specialties must possess current knowledge of obesity as an important, distinct disease with biological and social causes. Coverage of obesity on board certification examinations, which influence standards in medical knowledge and practice in each specialty, has not previously been examined. The member boards of the American Board of Medical Specialties offer a content outline or "blueprint" detailing material tested. We parsed the 24 available general certification exam blueprints for mentions of obesity and related keywords. We categorized blueprints into three tiers: mention of obesity (Tier 1), mention of related terminology but not obesity (Tier 2), and no mention of obesity or related terminology (Tier 3). We analyzed mentions of obesity and related terms by blueprint word count and procedural versus non-procedural specialties. Six (25.0%) of 24 board exam blueprints mentioned obesity (Tier 1), fifteen (62.5%) mentioned related terminology only (Tier 2), and three (12.5%) mentioned neither obesity nor related terminology (Tier 3). There was no significant difference in obesity-related mentions between procedural and non-procedural specialties (X2, p = .50). None of the blueprints included racial/ethnic disparities related to obesity. Word count was not significantly correlated with mentions of obesity in linear regression (p = .42). The absence of any mention of obesity on most content outlines and of racial/ethnic disparities on all content outlines indicates need for increased coverage of the diagnosis, prevention, and treatment of obesity across all board examinations.

Pearls and Pitfalls of the Virtual Interview: Perspectives From Both Sides of the Camera.

As the SARS-COV-2 pandemic created the need for social distancing and the implementation of nonessential travel bans, residency and fellowship programs have moved toward a web-based virtual process for applicant interviews. As part of the Society of Asian Academic Surgeons 5th Annual Meeting, an expert panel was convened to provide guidance for prospective applicants who are new to the process. This article provides perspectives from applicants who have successfully navigated the surgical subspecialty fellowship process, as well as program leadership who have held virtual interviews.

Is Initial Board Certification Associated With Better Early Career Surgical Outcomes?

OBJECTIVE: To determine if initial American Board of Surgery certification in general surgery is associated with better risk-adjusted patient outcomes for Medicare patients undergoing partial colectomy by an early career surgeon. BACKGROUND: Board certification is a voluntary commitment to professionalism, continued learning, and delivery of high-quality patient care. Not all surgeons are certified, and some have questioned the value of certification due to limited evidence that board-certified surgeons have better patient outcomes. In response, we examined the outcomes of certified versus noncertified early career general surgeons. METHODS: We identified Medicare patients who underwent a partial colectomy between 2008 and 2016 and were operated on by a non-subspecialty trained surgeon within their first 5 years of practice. Surgeon certification status was determined using the American Board of Surgery data. Generalized linear mixed models were used to control for patient-, procedure-, and hospital-level effects. Primary outcomes were the occurrence of severe complications and occurrence of death within 30 days. RESULTS: We identified 69,325 patients who underwent a partial colectomy by an early career general surgeon. The adjusted rate of severe complications after partial colectomy by certified (n = 4239) versus noncertified (n = 191) early-career general surgeons was 9.1% versus 10.7% (odds ratio 0.83, P = 0.03). Adjusted mortality rate for certified versus noncertified early-career general surgeons was 4.9% versus 6.1% (odds ratio 0.79, P = 0.01). CONCLUSION: Patients undergoing partial colectomy by an early career general surgeon have decreased odds of severe complications and death when their surgeon is board certified.

Truth-in-Advertising Laws: Are They Working? A Cross-Sectional Analysis of a "Plastic Surgeon" Patient Search Simulation.

BACKGROUND: Non-board-certified plastic surgeons performing cosmetic procedures and advertising as plastic surgeons may have an adverse effect on a patient's understanding of their practitioner's medical training and patient safety. The authors aim to assess (1) the impact of city size and locations and (2) the impact of health care transparency acts on the ratio of board-certified and non-American Board of Plastic Surgeons physicians. METHODS: The authors performed a systematic Google search for the term "plastic surgeon [city name]" to simulate a patient search of online providers. Comparisons of board certification status between the top hits for each city were made. Data gathered included city population, regional location, practice setting, and states with the passage of truth-in-advertising laws. RESULTS: One thousand six hundred seventy-seven unique practitioners were extracted. Of these, 1289 practitioners (76.9 percent) were American Board of Plastic Surgery-certified plastic surgeons. When comparing states with truth-in-advertising laws and states without such laws, the authors found no significant differences in board-certification rates among "plastic surgery" practitioners (88.9 percent versus 92.0 percent; p = 0.170). There was a significant difference between board-certified "plastic surgeons" versus out-of-scope practitioners on Google search between large, medium, and small cities (100 percent versus 92.9 percent versus 86.5; p < 0.001). CONCLUSIONS: Non-board-certified providers tend to localize to smaller cities. Truth-in-advertising laws have not yet had an impact on the way a number of non-American Board of Plastic Surgery-certified practitioners market themselves. There may be room to expand the scope of truth-in-advertising laws to the online world and to smaller cities.

Poisson Probability of Failing to Meet Minimum Case Volumes in Pediatric Surgery Fellowships.

INTRODUCTION: With the expansion of pediatric surgery fellowships from 2008 to 2018, there is concern for the dilution of training experience, especially for rare index cases. The Accreditation Council for Graduate Medical Education (ACGME) established required minimum case numbers by case type, but this is a program requirement rather than an individual trainee requirement. The American Board of Surgery (ABS) is considering instituting minimum case requirements across 5 broad categories for individuals to be board-eligible in pediatric surgery. METHODS: The ACGME National Data Report summary case logs were obtained for graduating fellows in pediatric surgery from 2008 to 2018. Median case volumes were compared to minimum ACGME case numbers and proposed ABS individual requirements. Using Poisson distributions, probabilities of individual fellows failing to meet minimum case numbers were calculated. RESULTS: The average annual probability that a median program would fail to meet minimum ACGME case numbers in at least 1 category was estimated at 16.6%. Using the proposed ABS system, the probability of failure was estimated at 44.1%. No temporal trend was found in the annual probability of failure in either the ACGME or the proposed ABS system. CONCLUSIONS: There is significant risk of a fellow failing to meet case minimums in the ACGME system and the proposed ABS system. This probability is increased for the half of programs below median. If the ABS institutes case minimums as a requirement for certification in pediatric surgery, the current training paradigm may be impacted at some programs.

Time Requirement and Feasibility of a Systematic Quality Peer Review of Reporting in Radiology.

OBJECTIVE: To estimate the human resources required for a retrospective quality review of different percentages of all routine diagnostic procedures in the Department of Radiology at Bern University Hospital, Switzerland. MATERIALS AND METHODS: Three board-certified radiologists retrospectively evaluated the quality of the radiological reports of a total of 150 examinations (5 different examination types: abdominal CT, chest CT, mammography, conventional X-ray images and abdominal MRI). Each report was assigned a RADPEER score of 1 to 3 (score 1: concur with previous interpretation; score 2: discrepancy in interpretation/not ordinarily expected to be made; score 3: discrepancy in interpretation/should be made most of the time). The time (in seconds, s) required for each review was documented and compared. A sensitivity analysis was conducted to calculate the total workload for reviewing different percentages of the total annual reporting volume of the clinic. RESULTS: Among the total of 450 reviews analyzed, 91.1 % (410/450) were assigned a score of 1 and 8.9 % (40/450) were assigned scores of 2 or 3. The average time (in seconds) required for a peer review was 60.4 s (min. 5 s, max. 245 s). The reviewer with the greatest clinical experience needed significantly less time for reviewing the reports than the two reviewers with less clinical expertise (p < 0.05). Average review times were longer for discrepant ratings with a score of 2 or 3 (p < 0.05). The total time requirement calculated for reviewing all 5 types of examination for one year would be more than 1200 working hours. CONCLUSION: A retrospective peer review of reports of radiological examinations using the RADPEER system requires considerable human resources. However, to improve quality, it seems feasible to peer review at least a portion of the total yearly reporting volume. KEY POINTS: · A systematic retrospective assessment of the content of radiological reports using the RADPEER system involves high personnel costs.. · The retrospective assessment of all reports of a clinic or practice seems unrealistic due to the lack of highly specialized personnel.. · At least part of all reports should be reviewed with the aim of improving the quality of reports.. CITATION FORMAT: · Maurer MH, Brönnimann M, Schroeder C et al. Time Requirement and Feasibility of a Systematic Quality Peer Review of Reporting in Radiology. Fortschr Röntgenstr 2021; 193: 160 - 167.