ABR examinations: the why, what, and how.

The American Board of Radiology (ABR) has provided certification for diagnostic radiologists and other specialists and subspecialists for more than 75 years. The Board certification process is a tangible expression of the social contract between the profession and the public by which the profession enjoys the privilege of self-regulation and the public is assured that it can expect medical professionals to put patients' interests first, guarantees the competence of practitioners, and guards the public health. A primary tool used by the ABR in fulfilling this responsibility is the secure proctored examination. This article sets forth seven standards based on authoritative sources in the field of psychometrics (the science of mental measurements), and explains in each case how the ABR implements that standard. Readers are encouraged to understand that, despite the multiple opinions that may be held, these standards developed over decades by experts using the scientific method should be the central feature in any discussion or critique of examinations given for the privilege of professional practice and for safeguarding the public well-being.

Addressing overutilization in medical imaging.

The growth in medical imaging over the past 2 decades has yielded unarguable benefits to patients in terms of longer lives of higher quality. This growth reflects new technologies and applications, including high-tech services such as multisection computed tomography (CT), magnetic resonance (MR) imaging, and positron emission tomography (PET). Some part of the growth, however, can be attributed to the overutilization of imaging services. This report examines the causes of the overutilization of imaging and identifies ways of addressing the causes so that overutilization can be reduced. In August 2009, the American Board of Radiology Foundation hosted a 2-day summit to discuss the causes and effects of the overutilization of imaging. More than 60 organizations were represented at the meeting, including health care accreditation and certification entities, foundations, government agencies, hospital and health systems, insurers, medical societies, health care quality consortia, and standards and regulatory agencies. Key forces influencing overutilization were identified. These include the payment mechanisms and financial incentives in the U.S. health care system; the practice behavior of referring physicians; self-referral, including referral for additional radiologic examinations; defensive medicine; missed educational opportunities when inappropriate procedures are requested; patient expectations; and duplicate imaging studies. Summit participants suggested several areas for improvement to reduce overutilization, including a national collaborative effort to develop evidence-based appropriateness criteria for imaging; greater use of practice guidelines in requesting and conducting imaging studies; decision support at point of care; education of referring physicians, patients, and the public; accreditation of imaging facilities; management of self-referral and defensive medicine; and payment reform.

American Board of Radiology diagnostic radiology initial qualifying (written) examinations.

The purpose of this article is to inform radiology residents, program directors, and other interested parties of the processes involved in developing, administering, and scoring the American Board of Radiology (ABR) diagnostic radiology qualifying (written) examinations. The residents, once certified, will have a lifelong professional relationship with the ABR. It is important for the ABR to be transparent about the processes it uses to ensure that its examinations are fair, valid, and reliable so that residents and their program directors have accurate information about these high-stakes examinations.

American Board of Radiology certifying examination: oral versus computer-based format.

OBJECTIVE: This pilot study of a computer-based examination for primary certification by the American Board of Radiology was designed to acquire comparative data on candidates that were measures of individual performance on the oral examination compared with the computer-based examination. MATERIALS AND METHODS: The pilot computer-based pediatric radiology examination was designed by experienced oral board examiners and the pediatric subspecialty trustees. Images were chosen from the examination repository of the American Board of Radiology. The 20-minute examination was designed to include 8-10 cases with 26-31 scorable units covering all aspects of pediatric radiology. RESULTS: Among the 1,317 candidates taking the oral board examination, 1,048 candidates (79.6%) participated in the voluntary pilot examination. The scores of the two examinations were subjected to statistical analysis. The sensitivity and specificity of the pilot examination were 94.5% and 45.7%. The overall accuracy was 92.8%. Seventy-five candidates (7.2%) who participated in this study received different verdicts on the pilot examination and the pediatric radiology category of the oral examination. Fifty-six of these candidates (5.3%) failed the pilot examination but passed in the oral pediatric radiology category; 19 of the candidates (1.8%) passed the pilot examination but failed the oral pediatric radiology test. Pilot examination scores were higher for candidates who passed the oral pediatric radiology category (median score, 80; interquartile range, 74.1-85.2) than for candidates who failed (median score, 65.4; interquartile range, 58.6-71.0) (p < 0.0001). CONCLUSION: The pediatric pilot examination was useful for differentiating passing candidates from failing candidates when the score in the pediatric radiology category of the oral examination was used as the reference standard. The overall accuracy was 92.8%.

American Board of Radiology Maintenance of Certification - Part IV: practice quality improvement for radiation oncology.

Maintenance of Certification is a physician-based response to public concerns about the quality of medical care and physician competency in a rapidly evolving, technically demanding specialty. The American Board of Radiology (ABR) has previously described the first three components of the Maintenance of Certification. The ABR is currently developing a program in practice performance, completing Part IV of the competencies. The Practice Quality Improvement (PQI) program is meant to critically evaluate meaningful aspects of a physician's practice in a simple manner, using identifiable metrics and self-assessment to include an action plan for quality improvement. Each diplomate will be expected to complete three PQI projects during a full 10-year Maintenance of Certification cycle. Current diplomates with time-limited certificates will find prorated requirements determined by their year of certification on the ABR Website. Diplomates will have the option of completing zero to two Type I PQI projects (assessing factors relevant to clinical practice by peer review and self-reporting) and one to three Type II projects (i.e., at least one Type II projects of the three required, assessing parameters of practice by comparison with evidence-based guidelines, consensus statements, or peer comparisons; Type II projects are initiated and managed by professional societies). Several examples of Type I projects that might be offered by societies or directly through the ABR are provided, as well as highlights of the two Type II projects that have sought ABR qualification: the American Society for Therapeutic Radiology and Oncology's Performance Assessment for the Advancement of Radiation Oncology Treatment program and American College of Radiology's RO-PEER program. An additional objective of the PQI is to develop national databases for future reference using aggregate data from the PQI projects.

The American Board of Radiology Perspective on Maintenance of Certification: Part IV: Practice quality improvement in radiologic physics.

Recent initiatives of the American Board of Medical Specialties (ABMS) in the area of maintenance of certification (MOC) have been reflective of the response of the medical community to address public concerns regarding quality of care, medical error reduction, and patient safety. In March 2000, the 24 member boards of the ABMS representing all medical subspecialties in the USA agreed to initiate specialty-specific maintenance of certification (MOC) programs. The American Board of Radiology (ABR) MOC program for diagnostic radiology, radiation oncology, and radiologic physics has been developed, approved by the ABMS, and initiated with full implementation for all three disciplines beginning in 2007. The overriding objective of MOC is to improve the quality of health care through diplomate-initiated learning and quality improvement. The four component parts to the MOC process are: Part I: Professional standing, Part II: Evidence of life long learning and periodic self-assessment, Part III: Cognitive expertise, and Part IV: Evaluation of performance in practice (with the latter being the focus of this paper). The key components of Part IV require a physicist-based response to demonstrate commitment to practice quality improvement (PQI) and progress in continuing individual competence in practice. Diplomates of radiologic physics must select a project to be completed over the ten-year cycle that potentially can improve the quality of the diplomate's individual or systems practice and enhance the quality of care. Five categories have been created from which an individual radiologic physics diplomate can select one required PQI project: (1) Safety for patients, employees, and the public, (2) accuracy of analyses and calculations, (3) report turnaround time and communication issues, (4) practice guidelines and technical standards, and (5) surveys (including peer review of self-assessment reports). Each diplomate may select a project appropriate for an individual, participate in a project within a clinical department, participate in a peer review of a self-assessment report, or choose a qualified national project sponsored by a society. Once a project has been selected, the steps are: (1) Collect baseline data relevant to the chosen project, (2) review and analyze the data, (3) create and implement an improvement plan, (4) remeasure and track, and (5) report participation to the ABR, using the template provided by the ABR. These steps begin in Year 2, following training in Year 1. Specific examples of individual PQI projects for each of the three disciplines of radiologic physics are provided. Now, through the MOC programs, the relationship between the radiologic physicist and the ABR will be continuous through the diplomate's professional career. The ABR is committed to providing an effective infrastructure that will promote and assist the process of continuing professional development including the enhancement of practice quality improvement for radiologic physicists.

Reliability of oral examinations: Radiation oncology certifying examination.

PURPOSE: Oral examinations are used in certifying examinations by many medical specialty boards. They represent daily clinical practice situations more realistically than do written tests or computer-based tests. However, there are repeated concerns in the literature regarding objectivity, fairness, and extraneous factors from interpersonal interactions, item bias, reliability, and validity. In this study, the reliability of oral examination on the radiation oncology certifying examination, which was administered in May of 2010, was analyzed. METHODS AND MATERIALS: One hundred fifty-two candidates rotated though 8 examination stations. Stations consisted of a hotel room equipped with a computer and software that exhibited images appropriate to the content areas. Each candidate had a 25-30 minute face-to-face encounter with an oral examiner who was a content expert in one of the following areas: gastrointestinal, gynecology, genitourinary, lymphoma/leukemia/transplant/myeloma, head/neck/skin, breast, central nervous system/pediatrics, or lung/sarcoma. This type of design is typically referred to as a repeated measures design or a subject by treatment design, although the oral examination was a routine event without any experimental manipulation. RESULTS: The reliability coefficient was obtained by applying Feldt and Charter's simple computational alternative to analysis of variance formulas that yielded KR-20, or Cronbach's coefficient alpha of 0.81. CONCLUSIONS: An experimental design to develop a blueprint in order to improve the consistency of evaluation is suggested.

ABR examinations: the why, what, and how.

The American Board of Radiology (ABR) has provided certification for diagnostic radiologists and other specialists and subspecialists for more than 75 years. The Board certification process is a tangible expression of the social contract between the profession and the public by which the profession enjoys the privilege of self-regulation and the public is assured that it can expect medical professionals to put patients' interests first, guarantees the competence of practitioners, and guards the public health. A primary tool used by the ABR in fulfilling this responsibility is the secure proctored examination. This article sets forth seven standards based on authoritative sources in the field of psychometrics (the science of mental measurements), and explains in each case how the ABR implements that standard. Readers are encouraged to understand that, despite the multiple opinions that may be held, these standards developed over decades by experts using the scientific method should be the central feature in any discussion or critique of examinations given for the privilege of professional practice and for safeguarding the public well-being.

Introduction to value-based payment modifiers.

Value-based payment modifiers were legislated by Congress in the 2010 Patient Protection and Affordable Care Act. It is clear in the legislation, and the corresponding proposals published by the secretary of the US Department of Health and Human Services in late 2011, that the intent is to move from paying physicians for reporting to paying physicians for performance. The proposals, developed jointly with CMS, specify that the calculation of payments for performance will be a composite of quality and cost measures. The base year for determining performance benchmarks for the performance measures will be 2013, and the measures will be applied to physician payments on a limited basis beginning in 2015 and to all physician payments by 2017. The role of medical specialty boards, such as the ABR, in the development and deployment of measures is highlighted in this context. CMS's recent conversations with board representatives have indicated their view that the boards' measure development activities are key to increasing physician (especially specialist) participation in the Physician Quality Reporting System to 50% by 2015, from 20% to 30% today. The ABR will continue its past activities in this arena, working with the American Board of Medical Specialties, CMS, and specialty societies, so that ABR diplomates will be able to simultaneously complete their Maintenance of Certification requirements, satisfy the requirements for CMS incentives, and avoid penalties.

Web modules on professionalism and ethics.

Health care disciplines have always held resolutely to a commitment to professionalism and high ethical standards. With the present emphasis on public accountability, professionalism and ethics are receiving enhanced attention in health care education and practice. A challenge for radiologists, radiation oncologists, and medical physicists is to define the scope and depth of knowledge about professionalism and ethics that are necessary for the practice of the disciplines. A further challenge is to develop accessible educational materials that encompass this required knowledge. About 2 years ago, the ABR Foundation decided to address these challenges through the development of an ethics and professionalism curriculum and production of a series of Web-based educational modules that follow the curriculum. Six organizations agreed initially to contribute financially to construction of the curriculum and modules and were later joined by a seventh. The curriculum was developed by the ABR Foundation and included in a request for proposals that was widely distributed. Teams of authors for each of 10 modules were selected from respondents to the request for proposals. As the modules were developed, they were reviewed in 3 successive stages, including peer review by members of the ACR Committee on Professionalism and the RSNA-ACR Task Force on an Ethics Curriculum. After revisions were prepared in response to the reviews, the modules were translated into a format compatible with the e-learning platform on which they are mounted. The modules are now available to all who wish to study them.

Practice analysis: a basis for content validity for american board of radiology examinations in diagnostic radiology.

The ABR performs practice analysis every 3 years, according to its strategic plan, in an effort to strengthen the content validity of its qualifying and certifying examinations as well as its maintenance of certification examinations. A nationwide survey of diagnostic radiologists was conducted in July 2010 for the purpose of determining the critically important and frequently performed activities in 12 clinical categories. The survey instrument was distributed electronically to 17,721 members of the ACR, with a unique identification code for each individual. A 5-point scale was established for both frequency and importance variables. The frequency scale ranged from 1 to 5 as follows: 1 = not applicable, 2 = occasionally, 3 = monthly, 4 = weekly, and 5 = daily. The scale for importance also ranged from 1 to 5: 1 = not applicable, 2 = not important, 3 = somewhat important, 4 = important, and 5 = essential. A total of 2,909 diagnostic radiologists (19.32%) participated. Of these, 2,233 (76.76%) indicated that they spent ≥50% of their time in clinical practice. Because of its brevity of the list of the activities, results for the gastrointestinal category are presented in this article. The list of activities weighted according to importance and frequency is presented in this article and, as illustrated, could become the foundation for developing a more detailed blueprint for the gastrointestinal category certifying examinations in diagnostic radiology. Findings on demographic information are also presented.

American Board of Radiology radiation oncology initial qualifying ("written") examinations.

This report was prepared by those who are closely involved in the radiation oncology initial qualification examinations. The primary purpose of this article is to disseminate information concerning test preparation, test administration, scoring, and reporting processes of the ABR. The authors hope that the information contained in the article will be helpful to radiology residents, program directors, and other interested parties.