Development and Results of the First Certification Examination in the American Board of Medical Specialties Neurocritical Care Subspecialty.

This article reviews the development of the American Board of Medical Specialties subspecialty in neurocritical care (NCC) and describes the requirements for certification and the results of the first certification examination administered in October 2021. The American Board of Psychiatry and Neurology (ABPN) is the administrative board, and the sponsoring boards are the American Board of Anesthesiology (ABA), American Board of Emergency Medicine (ABEM), American Board of Internal Medicine (ABIM), and American Board of Neurological Surgery. The American Board of Medical Specialties approved the subspecialty in 2018, and the Accreditation Council for Graduate Medical Education developed and approved the training requirements in 2021. The fellowship programs are either 12 or 24 months in length and may become available in Academic Year 2022-2023. The first NCC examination was developed by a multispecialty group of subject matter experts following established test development procedures and was successfully administered to 1,011 candidates in October 2021. There were 406 (40.2%) ABIM candidates, 356 (35.2%) ABPN candidates, 208 (20.6%) ABA candidates, and 41 (4.1%) ABEM candidates. The end-of-test survey indicated that most examinees were satisfied with their test taking experience, and the .92 reliability index indicated that the test scores were reliable. An established process was also followed to set the criterion-referenced passing standard, and the resulting pass rate of 72.7% was judged to be reasonable. In summary, the combined efforts of representatives from the ABPN, ABA, ABEM, ABIM, and American Board of Neurological Surgery yielded a quality assessment instrument to identify physicians who possess the expertise required to be certified in NCC. The test development committee will continue to expand and improve the pool of test questions for the next examination, which is scheduled for October 2022.

The Development and Current Status of Subspecialty Training and Certification in Addiction Psychiatry.

OBJECTIVE: This article describes the evolution of subspecialty training and certification in addiction psychiatry. The impact of the newer subspecialty in addiction medicine is also addressed. METHODS: Information about programs and trainees was obtained from records of the Accreditation Council for Graduate Medical Education. Information about addiction psychiatry certification was obtained from the records of the American Board of Psychiatry and Neurology (ABPN). Information about the addiction medicine subspecialty was obtained from the American Board of Preventive Medicine. RESULTS: In AY 2020-2021, there were 53 addiction psychiatry programs with 92 fellows, and the numbers of each have increased over the past 5 academic years. The total number of addiction psychiatry certificates awarded through 2020 was 2806. Three years after addiction medicine programs were first accredited, there were 83 programs with 149 fellows. Thus far, 3282 addiction medicine certificates have been awarded, 1275 (38.8%) of them to ABPN diplomates. CONCLUSIONS: In the 30 years since addiction psychiatry received subspecialty recognition, the numbers of training programs and fellows have grown steadily and are continuing to increase. Recently, the numbers of training programs and fellows in the newer subspecialty of addiction medicine have grown rapidly with substantial psychiatry involvement in addiction medicine training and certification programs. Nonetheless, it is apparent that the need for specialists with expertise in substance use disorders will far exceed the supply for the foreseeable future.

Primary Care and Neurology in Psychiatry Residency Training: a Survey of Early Career Psychiatrists.

OBJECTIVE: A survey of recently certified psychiatrists was conducted to obtain their feedback about the contribution of the primary care and neurology components of residency training to their professional development and to their current needs as practitioners. METHODS: A 22-item survey was developed based on issues discussed at a forum on residency competence requirements and administered electronically to four cohorts of recently certified psychiatrists. RESULTS: The response rate was 17% (1049/6083). Overall, the respondents described both their primary care and neurology experiences as helping them accomplish several goals for their professional development. The majority were satisfied with their primary care training and felt well-prepared to enter practice. The most common suggestions for improving the primary care component were better integration with psychiatry and providing longitudinal experiences and more outpatient experience. They were somewhat less satisfied with their neurology training, and only about half felt well-prepared for the neurologic aspects of psychiatry practice. The most common suggestions for improving neurology training were to provide more time in neurology with experiences that were more relevant to psychiatry such as outpatient and consultation experiences. Some also thought longitudinal experiences would be useful. CONCLUSIONS: These psychiatrists were generally satisfied with the primary care and neurology components of residency training and felt that they had contributed to their professional development. Their suggestions for improvement contribute to the rich discussion among training directors and other psychiatry educators about these components of residency training.

Subspecialty Training and Certification in Geriatric Psychiatry: A 25-Year Overview.

The Institute of Medicine estimated that by 2030, from 10.1 to 14.4 million Americans aged 65 years or older will have mental health or substance use disorders. This article reviews the history and current status of training, certification, and practice in geriatric psychiatry against the backdrop of this "silver tsunami." The American Board of Psychiatry and Neurology (ABPN) administered the first subspecialty examination in geriatric psychiatry in 1991, and through 2015 3,329 certificates were awarded. The Accreditation Council for Graduate Medical Education approved the training requirements in 1993. After a surge in programs and fellows, the numbers appear to have stabilized at about 57 programs and 60-65 trainees per year with fewer than half of the positions filled each year. The majority of graduates seeks and obtains ABPN certification, and the majority of those who were fellowship trained have maintained certification. Despite the unprecedented demand for mental health services for older adults, it must be acknowledged that not enough geriatric psychiatrists can be prepared to meet the needs of an aging U.S. POPULATION: Strategies for addressing the shortage are discussed, including undertaking subspecialty training in the fourth year of psychiatry training, increasing the time devoted to the care of older adults in undergraduate and graduate medical education, and developing alternative training pathways such as mini-fellowships. It is not clear whether more favorable Medicare reimbursement rates for those certified in geriatric psychiatry would increase the numbers seeking fellowship training.

Technical skills assessment toolbox: a review using the unitary framework of validity.

OBJECTIVE: The purpose of this study was to create a technical skills assessment toolbox for 35 basic and advanced skills/procedures that comprise the American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) surgical skills curriculum and to provide a critical appraisal of the included tools, using contemporary framework of validity. BACKGROUND: Competency-based training has become the predominant model in surgical education and assessment of performance is an essential component. Assessment methods must produce valid results to accurately determine the level of competency. METHODS: A search was performed, using PubMed and Google Scholar, to identify tools that have been developed for assessment of the targeted technical skills. RESULTS: A total of 23 assessment tools for the 35 ACS/APDS skills modules were identified. Some tools, such as Operative Performance Rating System (OSATS) and Objective Structured Assessment of Technical Skill (OPRS), have been tested for more than 1 procedure. Therefore, 30 modules had at least 1 assessment tool, with some common surgical procedures being addressed by several tools. Five modules had none. Only 3 studies used Messick's framework to design their validity studies. The remaining studies used an outdated framework on the basis of "types of validity." When analyzed using the contemporary framework, few of these studies demonstrated validity for content, internal structure, and relationship to other variables. CONCLUSIONS: This study provides an assessment toolbox for common surgical skills/procedures. Our review shows that few authors have used the contemporary unitary concept of validity for development of their assessment tools. As we progress toward competency-based training, future studies should provide evidence for various sources of validity using the contemporary framework.

An Article-Based Format for Medical Specialty and Subspecialty Maintenance of Certification.

INTRODUCTION: This article describes an article-based alternative for maintenance of certification that the American Board of Psychiatry and Neurology developed and began pilot testing in 2019. The rationale for and components of the pilot program are presented along with data on participant performance and feedback from the first year of implementation in three primary specialties (neurology, child neurology, and psychiatry) and one subspecialty (child and adolescent psychiatry). METHODS: Evaluation of the pilot program was guided by a widely used validity framework. Data were collected that addressed the five categories of validity evidence: content, response process, internal structure, relation to other variables, and consequences. RESULTS: Enrollment ranged from 66.7% for psychiatrists to 75.3% for child neurologists. For the 2019 cohort, the pass rates ranged from 92.6% for child and adolescent psychiatry to 98.7% for neurology, and very small numbers of diplomates failed or did not complete the process. For psychiatrists, there was a modest, but significant, relationship between performance on previous and subsequent maintenance of certification examinations. Ninety percent or more agreed that: the articles were easy to access and helpful to their practices; the mini-tests were a fair assessment of their understanding of the articles; and their test-taking experience was satisfactory. DISCUSSION: Most eligible diplomates participated in the article-based pilot project, and they strongly preferred this format to the traditional multiple-choice examinations. Most important, the pilot was perceived to be a meaningful and relevant learning activity that had a positive effect on patient care.

Recent trends in american board of psychiatry and neurology psychiatric subspecialties.

OBJECTIVE: this article reviews the current status and recent trends in the American Board of Psychiatry and Neurology (ABPN) psychiatric subspecialties and discusses the implications of those trends as well as several key questions whose answers may well determine subspecialty viability. METHODS: data are presented on specialty and subspecialty programs; graduates; and ABPN certification candidates and diplomates drawn from several sources, including the records of the ABPN, the websites of the Accreditation Council for Graduate Medical Education and the American Medical Association, and the annual medical education issues of JAMA. RESULTS: fewer than half of psychiatry graduates pursue subspecialty training. While most recent specialty graduates attempt to become certified by the ABPN, many subspecialists elect not to do so. There have been recent decreases in the number of fellowship programs and trainees in geriatric psychiatry and addiction psychiatry. The pass rates for fellowship graduates are superior to those for the "grandfathers" in all of the newer psychiatric subspecialties. Lower percentages of subspecialists than specialists participate in maintenance of certification, and maintenance of certification pass rates are high. CONCLUSION: the initial interest in training and certification in some of the ABPN subspecialties appears to have slowed, and the long-term viability of those subspecialties may well depend on the answers to a number of complicated social, economic, and political questions in the new health care era.

Training in Neurology: Feedback from Graduates About the Psychiatry Component of Residency Training.

OBJECTIVE: To obtain feedback from early career adult and pediatric neurologists about the psychiatry component of residency training. METHODS: A survey was developed and administered electronically to 4 cohorts of recently certified American Board of Psychiatry and Neurology diplomates. RESULTS: The response rate was 16% (431/2,677) and included 330 adult neurologists and 101 pediatric neurologists. Fewer than half of the respondents described themselves as extremely or quite satisfied with their psychiatry training whereas 26% of the adult neurologists and 33% of the pediatric neurologists felt slightly or not at all prepared for this component of practice. Four themes were identified in the respondents' suggestions for improving psychiatry training: provide more outpatient experience; provide more time/teaching in psychiatry; provide more experience with both pharmacologic and nonpharmacologic psychiatric treatments; and provide more exposure to patients with conditions likely to be encountered in neurology/child neurology practice. CONCLUSION: These recent graduates of adult and pediatric neurology residency programs felt underprepared for the psychiatric issues they encountered in their patients. They suggested a number of strategies for better alignment of psychiatry training with the likely demands of practice. A model curriculum recently developed by the American Academy of Neurology's Consortium of Neurology Program Directors and the American Association of Directors of Psychiatric Residency Training also provides guidance for both neurology and psychiatry program directors.

Subspecialization in clinical neurophysiology: Development and current status.

OBJECTIVE: To describe the development and current status of training and certification in clinical neurophysiology (CNP); to explore the impact of the newer subspecialties in sleep medicine, neuromuscular medicine, and epilepsy; and to obtain information about aspects of practice in the subspecialty. METHODS: Information about training programs and certification was obtained from the records of the Accreditation Council for Graduate Medical Education and the American Board of Psychiatry and Neurology, and diplomates were surveyed about their CNP practice activities and attitudes toward certification/recertification. RESULTS: In the years since the first examination was administered, a robust number of CNP training programs developed, but recently, there has been a decrease in the number of programs and fellows, although the number of programs and fellows in the subspecialties of epilepsy, neuromuscular medicine, and sleep medicine has increased. A diplomate survey indicated that most respondents devoted significant practice time to CNP procedures, especially to EEGs and EMGs. Although more diplomates performed EEGs than EMGs, a substantial portion performed both. Most diplomates were planning to or had maintained certification in CNP. CONCLUSION: Over 3,000 neurologists, child neurologists, and psychiatrists have obtained certification in CNP, and the majority are participating in recertification. Although the newer and overlapping subspecialties of epilepsy, neuromuscular medicine, and sleep medicine may be having a negative impact on CNP, it continues to have a relatively large number of programs and attracts a relatively large number of fellows.

Clinical Skills Evaluation in Neurology, Child Neurology and Psychiatry: Program Directors' Perspective and Future Opportunities.

BACKGROUND: When the American Board of Psychiatry and Neurology (ABPN) eliminated the oral segment of the board-certification examination, it began requiring in-training assessments termed Clinical Skill Evaluations (CSEs). OBJECTIVE: This study describes the experience of residency program directors (PDs) with CSEs and identifies opportunities for improvement. METHODS: A 23-question survey was administered electronically to neurology, child neurology, and psychiatry PDs assessing their CSE testing procedures in April 2019. Data from the ABPN preCERT® Credentialing System CSE was analyzed to corroborate the survey results. RESULTS: A total of 439 PDs were surveyed. The overall response rate was approximately 40% with a similar response across the 3 specialties. Overall, there was a strong enthusiasm for CSEs as they captured the essence of the physician-patient relationship. Most PDs encouraged trainees to attempt CSEs early in their training though the completion time frame varied by specialty. Approximately 57% of psychiatry residencies offered formal, in-person faculty training while less than one-fourth of neurology and child neurology programs offered such a program. Most PDs are interested in a faculty development course to ensure a standardized CSE testing process at their institution. CONCLUSIONS: This survey confirmed earlier findings that CSEs are usually implemented early in the course of residency training and that most PDs think it captures the essence of the physician-patient relationship. While few residencies offer a CSE training course, there is widespread support for a formal approach to faculty development and this offers a specific opportunity for CSE improvement in the future.

Relationship between Resident-In-Training Examination in psychiatry and subsequent certification examination performances.

OBJECTIVE: This study analyzed the relationship between performance on The American College of Psychiatrists' Psychiatry Resident-In-Training Examination (PRITE) and the ABPN Part 1 examination. METHODS: Pearson correlation coefficients were used to examine the relationship between performance on the 2002 PRITE and the 2003 Part 1 examination for 297 examinees. RESULTS: The correlation between the PRITE global psychiatry and the Part 1 psychiatry scores was 0.59, and the correlation between the PRITE global neurology and the Part 1 neurology scores was 0.39. CONCLUSION: Although the PRITE and the Part 1 examination have different purposes and are developed independently, the significant correlations between scores on the two tests support the use of PRITE results to guide preparation for the Part 1 examination. Guidelines for PRITE scores associated with poor performance on the Part 1 examination are provided.

Performance of Certification and Recertification Examinees on Multiple Choice Test Items: Does Physician Age Have an Impact?

INTRODUCTION: The development of recertification programs (now referred to as Maintenance of Certification or MOC) by the members of the American Board of Medical Specialties provides the opportunity to study knowledge base across the professional lifespan of physicians. Research results to date are mixed with some studies finding negative associations between age and various measures of competency and others finding no or minimal relationships. METHODS: Four groups of multiple choice test items that were independently developed for certification and MOC examinations in psychiatry and neurology were administered to certification and MOC examinees within each specialty. Percent correct scores were calculated for each examinee. Differences between certification and MOC examinees were compared using unpaired t tests, and logistic regression was used to compare MOC and certification examinee performance on the common test items. RESULTS: Except for the neurology certification test items that addressed basic neurology concepts, the performance of the certification and MOC examinees was similar. The differences in performance on individual test items did not consistently favor one group or the other and could not be attributed to any distinguishable content or format characteristics of those items. DISCUSSION: The findings of this study are encouraging in that physicians who had recently completed residency training possessed clinical knowledge that was comparable to that of experienced physicians, and the experienced physicians' clinical knowledge was equivalent to that of recent residency graduates. The role testing can play in enhancing expertise is described.

Comparison of Certification and Recertification Examinee Performance on Multiple-Choice Items in Forensic Psychiatry.

Research on the association between age and performance on tests of medical knowledge has generally shown an inverse relationship, which is of concern because of the positive association between measures of knowledge and measures of clinical performance. Because the certification and maintenance of certification (MOC) examinations in the subspecialty of forensic psychiatry draw on a common item bank, performance of the two groups of examinees on the same items could be compared. In addition, the relationship between age and test performance was analyzed. Performance on items administered to certification and MOC examinees did not differ significantly, and the mean amount of time spent on each item was similar for the two groups. Although the majority (five of eight) of the correlations between age and test score on the certification and MOC examinations were negative, only three were significant, and the amount of variance explained by age was small. In addition, examination performance for those younger than 50 was similar to those 60 and older, and diplomates recertifying for the second time outperformed those doing so for the first time. These results indicate that in this subspecialty, there is no clear evidence of an age-related decline in knowledge as assessed by multiple-choice items.

Validity and feasibility of the EMG direct observation tool (EMG-DOT).

OBJECTIVE: To develop a new workplace-based EMG direct observation tool (EMG-DOT) and gather validity evidence supporting its use for assessing electrodiagnostic skills among postgraduate medical trainees. METHODS: The EMG-DOT was developed by experts using an iterative process. Validity evidence from content, response process, internal structure, relations to other variables, and consequences of testing was collected during the 2013-2014 academic year. RESULTS: Of 3,412 studies performed by trainees during the study period, 299 (9%) were assessed using the EMG-DOT. Of these, 203 (68%) involved a physician rater and 96 (32%) involved a technician rater. The 14-item EMG-DOT had excellent internal-consistency reliability (Cronbach α 0.94). Correlations between individual items and criterion-referenced global ratings of performance ranged from 0.36 to 0.72 (all p < 0.001). Mean total scores increased from 70% to 80% over 4 months of the EMG rotation (p < 0.001) despite a corresponding significant increase in case complexity (0.21-0.74 on a 3-point rating scale; p < 0.001). Trainees reported that the observational assessment exercise improved their knowledge or skills in 82% of encounters (188/230) and that feedback generated by the EMG-DOT improved the quality of care provided to patients in 58% (133/230). Trainees were "satisfied" or "very satisfied" with the observational assessment exercise in 96% of encounters (234/243). CONCLUSIONS: This study provides validity evidence supporting the use of EMG-DOT scores to assess electrodiagnostic skills of residents and fellows. The EMG-DOT can be used to inform milestone-based assessments of trainee performance in neurology, child neurology, physical medicine and rehabilitation, neuromuscular, and clinical neurophysiology training programs.

Certification in child neurology: new directions for the twenty-first century.

The American Board of Psychiatry and Neurology was formed in 1934, and the first certifying examination was administered in 1935. It was not until 1969 that the American Board of Psychiatry and Neurology began conducting certifying examinations in child neurology. Since then, 10 child neurologists have served as American Board of Psychiatry and Neurology directors and nearly 1500 child neurologists have been certified, more than 200 of whom have also attained certification in the subspecialties of clinical neurophysiology, neurodevelopmental disabilities, and pain medicine. This article describes the evolution of the certification process as the Board has endeavored to use the best testing methodologies to fulfill its commitment to the public and to its constituents in psychiatry and neurology. Training pathways in child neurology are also reviewed as they have evolved in response to evident staff shortages and to promote entry into academic careers.

Board certification in child neurology and neurology: cohort study.

This study followed the progress of a cohort of child neurology and neurology graduates who sought certification by the American Board of Psychiatry and Neurology (ABPN). The 211 candidates were recent graduates when they initiated the certification process in October 1994, and 92% of them have achieved certification. First-attempt performances on the Part I and Part II examinations were related; 62% of those certified passed both on their first attempts. Of those who were certified, 24% also achieved certification in at least one additional ABPN specialty (psychiatry) or subspecialty (clinical neurophysiology, neurodevelopmental disabilities, and pain medicine). Overall, it was estimated that 82% of the 1994 child neurology and neurology graduates sought ABPN certification, and of these, 82% were successful.

Clinical Skills Assessment: The Effects of Moving Certification Requirements Into Neurology, Child Neurology, and Psychiatry Residency Training.

BACKGROUND: A few years ago, when the American Board of Psychiatry and Neurology decided to phase out the patient-based oral examinations in its 3 primary specialties, requirements for assessing clinical skills during residency training were instituted. OBJECTIVE: The purpose of this report is to describe the experiences of training program directors and graduates with these new credentialing requirements (labeled CSEs) as well as other effects on the specialties. METHODS: Surveys were administered electronically in 2012 to all current neurology, child neurology, and psychiatry program directors, and to a convenience sample of graduates who applied for the 2012 certification examinations. RESULTS: Response rates for graduates were similar across the 3 specialties but low (28%-33%). Response rates were higher for program directors (53%-62%) and were similar across the 3 specialties. The results indicated that the CSEs were usually administered early in training, were completed toward the end, were often passed on first attempt, generally took place during routine clinical assignments, were used to assess additional competencies, almost always included feedback to the residents, and did not often lead to remediation. Furthermore, the CSEs were perceived to be useful components in the assessment of clinical skills. CONCLUSIONS: The results obtained from the early implementation of the CSEs suggest that they provide an opportunity to assess clinical skills with the additional benefit of feedback to trainees. Other effects included eventual incorporation into training program requirements, milestones, and related faculty development and research efforts.

Achieving board certification in psychiatry: a cohort study.

OBJECTIVE: The purpose of this study was to track the progress of a cohort of graduates of psychiatry residency training programs in achieving certification by the American Board of Psychiatry and Neurology (ABPN). These data provide a detailed picture of how recent graduates perform on the ABPN's examinations. METHOD: The subjects for this study were the 739 new candidates who sat for the part I examination in fall 1994. The cohort's performance on the part I and part II examinations was analyzed, as was the relationship between performance on the two examinations. RESULTS: Approximately 8 years after their first attempt at the part I examination, 85% of the cohort were certified, and 15% were not. The majority passed both the part I and part II examinations on the first attempt. Those who passed part I on the first attempt were more likely than those who failed to pass part II on the first attempt. Of the 627 who were certified at follow-up, 199 (32%) were also certified in one subspecialty, and 29 (5%) were certified in two, for a total of 257 subspecialty certificates. CONCLUSIONS: The results of this study suggest that most recent graduates of residency training programs who attempt the ABPN process are likely to become board certified, and the majority will do so by passing both components on the first attempt.

RITE performance predicts outcome on the ABPN Part I examination.

OBJECTIVE: This study examined the performance correlation the American Academy of Neurology Resident Inservice Training Examination (RITE) and Part I of the American Board of Psychiatry and Neurology (ABPN) examination. METHODS: Candidates who took the RITE in the last year of training were tracked and their subsequent performance on the ABPN Part I examination was compared with their performance on the RITE. RESULTS: Performance on the RITE is predictive of performance on the ABPN Part I examination. CONCLUSIONS: The RITE can be used to assess a resident's readiness to take the ABPN Part I examination.