Can professionalism be measured? The development of a scale for use in the medical environment.

PURPOSE: To assess a scale that measures professional attitudes and behaviors associated with the medical education and the residency training environment. METHOD: In 1995-96, the authors surveyed medical students and residents from five institutions in the northeast region of the United States. RESULTS: Of 757 distributed questionnaires, 565 were returned (75% response rate). Of those, 529 (94%) were used in the analysis. The mean score for the retained 12 items was 92.9 (SD, 11.9), with higher scores indicating more positive perceptions. The internal reliability of the scale was moderately high (alpha = .71). A factor analysis identified three subscales: excellence, honor/integrity, and altruism/respect, with eigenvalues (alpha coefficients) of 3.18 (.72), 1.70 (.60), and 1.20 (.59), respectively. CONCLUSIONS: The results are interpreted as an encouraging first step toward the development of a reliable scale that measures professionalism within the environment of medical education and residency training.

Overview on ABIM End-of-Life Patient Care Project: caring for the dying: identification and promotion of physician competency.

The American Board of Internal Medicine's project to improve end-of-life care was initiated in response to lack of attention given to death and dying in the United States. The project focuses mainly on physician competency in residency and fellowship training. With this data, recommendations can be made to improve physician training in end-of-life care treatment.

Examiner differences in the mini-CEX.

OBJECTIVE: The objective of this study was to analyze whether faculty ratings of residents, using the mini-CEX oral exam format, differed in stringency or were influenced by the clinical setting. It also sought to learn whether the examiners were satisfied with the format. METHOD: A mini-CEX encounter consisted of a single faculty member observing a resident conduct a focused history and physical examination in an inpatient, outpatient, or emergency room setting. After asking the resident for a diagnosis and treatment plan, the faculty member rated the resident and provided educational feedback. The encounters were intended to be short and occur as a routine part of the training, so each resident would be evaluated on many occasions by different faculty. SAMPLE: Sixty-four attending physicians evaluated residents from five internal medicine training programs; data were analyzed for 355 mini-CEX encounters involving 88 residents. RESULTS: There were not large differences among the examiners in their ratings. Moreover, there were not great differences among the ratings in terms of the training program with which the examiner was associated, the setting of the mini-CEX, or the nature of the patient. The examiners were generally satisfied with the format and their level of satisfaction was correlated with the residents' perceptions of the format. CONCLUSION: The mini-CEX adapts itself to a broad range of clinical situations, and these results show that it should produce roughly comparable scores over examiners and settings. This makes it a worthwhile device for evaluation at the local level.

Refocusing on history-taking skills during internal medicine training.

Recognizing that skilled history-taking is in danger of becoming a lost art, the American Board of Internal Medicine calls attention to the urgent need for internal medicine residency programs to ensure that these skills are taught and assessed. Although the Board's certification examination contains standardized items that test the physician's ability to use information from a patient's medical history, the written examination cannot assess the physician's ability to elicit that history. The Board believes that history-taking skills will become even more crucial as health care delivery changes, requiring more cost efficiency without sacrificing quality. By highlighting the skills of effective history-taking and strategies for assessment, the Board offers specific recommendations for its promotion as a key element of quality patient care.

Feasibility of hospital-based use of peer ratings to evaluate the performances of practicing physicians.

PURPOSE: To address the feasibility of obtaining reliable evaluations of individual physicians from peer ratings undertaken at diverse hospitals. METHOD: Eleven hospitals in diverse locations in the United States were recruited to participate. With the aid of the hospitals' medical directors, up to 40 board-certified internists with admitting privileges were recruited per hospital. Participating physicians provided demographic data about themselves and nominated physician-associates to do peer ratings. Between April 1993 and January 1994, the physicians were rated by their peers, who received a single mailing with no follow-up. The raters used a nine-point Likert scale for 11 cognitive and noncognitive categories. Administrative procedures were coordinated from the American Board of Internal Medicine. Chi-square, Student's t-test, and factor analysis using varimax rotation were used to analyze the results. RESULTS: Of the 4,139 questionnaires that were mailed to peer raters, 3,005 (73%) were returned. Of the 228 physicians who were rated, 187 received ten or more usable ratings, which were used for further analysis. The findings confirmed the results of previous research. The highest mean rating was for the category of integrity, and the lowest was for the category of psychosocial aspects of care. Ten to 11 responses per physician were necessary to achieve a generalizability coefficient of .7. Nearly 90% of the variance in the ratings was accounted for by two factors, one representing cognitive and clinical management skills and the other, humanistic qualities. For 16 physicians (9%), the ratngs of overall clinical skills were less than 7 on a scale from 1 (low) to 9 (high); their ratings for all individual cognitive and noncognitive categories were below the ratings of the other physicians. CONCLUSION: The peer raters' response rate and the analysis of the ratings suggest that the rating process is acceptable to physicians and that it is feasible to obtain reliable, multidimensional peer evaluations of individual physicians practicing in diverse clinical settings.

The mini-CEX (clinical evaluation exercise): a preliminary investigation.

OBJECTIVE: To gather preliminary data on the mini-CEX (clinical evaluation exercise), a device for assessing the clinical skills of residents. DESIGN: Evaluation of residents by faculty members using the mini-CEX. SETTING: 5 internal medicine training programs in Pennsylvania. PARTICIPANTS: 388 mini-CEX encounters involving 88 residents and 97 evaluators. MEASUREMENTS: A mini-CEX encounter consists of a single faculty member observing a resident while that resident conducts a focused history and physical examination in any of several settings. After asking the resident for a diagnosis and treatment plan, the faculty member rates the resident and provides educational feedback. The encounters are intended to be short (about 20 minutes) and to occur as a routine part of training so that each resident can be evaluated on several occasions by different faculty members. RESULTS: The encounters occurred in both inpatient and ambulatory settings and were longer than anticipated (median duration, 25 minutes). Residents saw either new or follow-up patients who collectively presented with a broad range of clinical problems. The median evaluator assessed two residents and was generally satisfied with the mini-CEX format; residents were even more satisfied with the format. The reproducibility of the mini-CEX is higher than that of the traditional CEX, and its measurement characteristics are similar to those of other test formats, such as standardized patients and standardized oral examinations. CONCLUSIONS: The mini-CEX assesses residents in a much broader range of clinical situations than the traditional CEX, has better reproducibility, and offers residents greater opportunity for observation and feedback by more than one faculty member and with more than one patient. On the other hand, the mini-CEX may be more difficult to administer because multiple encounters must be scheduled for each resident. Exclusive use of the mini-CEX also prevents residents from being observed while doing a complete history and physical examination. Given the promising results and measurement characteristics of the mini-CEX, however, the American Board of Internal Medicine encourages the use of this method in conjunction with or as an alternative to the traditional CEX.

Procedural skills of practicing pulmonologists. A national survey of 1,000 members of the American College of Physicians.

We surveyed pulmonologists to determine which procedures they do in practice, where they learned the procedures, and how much training they recommend to attain and maintain clinical competence in each. We mailed a survey to a random sample of 1,000 members of the American College of Physicians who were identified as practicing pulmonologists; 755 (75%) responded. Respondents performed a variety of pulmonary procedures, an average of 17 of the 29 listed. Pulmonologists who were more recent graduates, who worked longer hours, and who were involved in critical care did a greater variety of procedures. Only 26% of practicing pulmonologists currently do all the procedures required for board certification in pulmonary medicine. For each of 13 specific procedures, the number reported done in the past year was generally unrelated to practice factors. Many respondents who learned procedures in practice did so without formal training or supervision. Respondents' recommendations regarding numbers of procedures required to attain or maintain competence did not vary greatly. Pulmonologists vary considerably in the types of procedures they do. Their opinions about the training needed for competence help to better define requirements for training programs. More attention should be focused on training and certifying practicing pulmonologists in procedures learned after formal fellowship training.

Procedural skills of practicing gastroenterologists. A national survey of 700 members of the American College of Physicians.

OBJECTIVE: To find which procedures gastroenterologists do in their practice, where they learned the procedures, and what training they recommend to achieve and maintain competence in each. DESIGN: Mailed survey. PARTICIPANTS: A random sample of 700 gastroenterologists who were members of the American College of Physicians, of whom 510 (73%) responded. RESULTS: Gastroenterologists did a median of 27 of the 40 gastroenterology procedures and 3 of the 16 general medicine procedures. A larger number of different procedures was done by those who were board certified in gastroenterology, spent more than 60 hours per week in patient care, graduated after 1969, or practiced in a university setting, a hospital with 250 to 500 beds, or a group limited to gastroenterologists. The majority of respondents had done each of the three most commonly done procedures 200 or more times in the previous year. For 10 procedures studied in detail, the majority had done 9 of the 10 in the past year at least as often as the median recommendation for maintaining competence. The number of procedures recommended to achieve and maintain competence differed little by the age, type of practice, and gastroenterology board certification status of the respondent. For most procedures, the range of recommendations was broad, but the median numbers agreed with other published recommendations. Respondents learned 31% of procedures after they had completed training. The earlier their training, the more likely they had learned the procedure in practice. CONCLUSIONS: Gastroenterologists do many different procedures, and this number is influenced by how recently they were trained and the nature of their practice. Respondents learned nearly one third of the procedures they do after they had completed training. It is an important and continuing challenge for the profession to assure that physicians learn and maintain these procedural skills at the highest level of proficiency.

Residents' perception of evaluation procedures used by their training program.

OBJECTIVE: To determine the methods of evaluation used routinely by training programs and to obtain information concerning the frequencies with which various evaluation methods were used. DESIGN: Survey of residents who had recently completed internal medicine training. PARTICIPANTS: 5,693 respondents who completed residencies in 1987 and 1988 and were registered as first-time takers for the 1988 Certifying Examination in Internal Medicine. This constituted a 76% response rate. MAIN RESULTS: Virtually all residents were aware that routine evaluations were submitted on inpatient rotations, but were more uncertain about the evaluation process in the outpatient setting and the methods used to assess their humanistic qualities. Most residents had undergone a Clinical Evaluation Exercise (CEX); residents' clinical skills were less likely to be evaluated by direct observation of history or physical examination skills. Resident responses were aggregated within training programs to determine the pattern of evaluation across programs. The majority of programs used Advanced Cardiac Life Support (ACLS) certification, medical record audit, and the national In-Training Examination to assess most of their residents. Performance-based tests were used selectively by a third or more of the programs. Breast and pelvic examination skills and ability to perform sigmoidoscopy were thought not to be adequately assessed by the majority of residents in almost half of the programs. CONCLUSIONS: While most residents are receiving routine evaluation, including a CEX, increased efforts to educate residents about their evaluation system, to strengthen evaluation in the outpatient setting, and to evaluate certain procedural skills are recommended.

Procedural skills of practicing nephrologists. A national survey of 700 members of the American College of Physicians.

OBJECTIVE: To determine which procedures nephrologists do in practice, where they learned the procedures, and how much training they recommend to achieve and maintain clinical competence in each. DESIGN: Mailed survey. PARTICIPANTS: A random sample of 700 members of the American College of Physicians who were identified as practicing nephrologists; 516 (74%) responded. RESULTS: Acute peritoneal dialysis, acute hemodialysis, continuous arteriovenous hemofiltration, and percutaneous renal biopsy were done by 95%, 97%, 87%, and 91% of the respondents, respectively. Except for hemofiltration, procedures were learned by most respondents during fellowship training. Compared with general internists, fewer nephrologists did most of the 19 general procedures included in the survey. The number of nephrology procedures done during the past year varied considerably among respondents, as did the minimum number of procedures they recommended for achieving and maintaining competence. Neither the variation in number of procedures done nor recommendations regarding certification were explained by differences in practice characteristics. Median recommendations remained relatively constant among subgroups. CONCLUSIONS: Nearly all nephrologists do the four nephrology procedures included in the survey. Their opinions about the training needed for competence help to better define requirements for training programs. More attention should be focused on training and certifying practicing nephrologists in procedures learned after formal fellowship training.

Procedural skills of the general internist. A survey of 2500 physicians.

STUDY OBJECTIVE: To determine which of 40 clinical procedures general internists do in their practice, how often these procedures are done, and what training is needed to develop and maintain competence in each. DESIGN: Mailed survey. PARTICIPANTS: A random sample of 2500 American College of Physicians (ACP) members identified as general internists; 1806 (72%) responded. Of these, we excluded 398 who were board eligible or board certified in a subspecialty and 143 with unknown status. The resulting sample of 1179 was augmented in selected analyses by an additional sample of 199 rural internists. RESULTS: General internists did, on average, 16 of the 40 procedures. Practice characteristics markedly affected the number and variety of procedures done. A larger number of different procedures was independently associated with smaller cities, smaller hospitals, younger age, increased hours in patient care, and certain regions of the country. Practice characteristics varied considerably by practice type and location. Many procedures were done infrequently. There was reasonable agreement on the experience needed to attain and maintain competency in each procedure. The majority of respondents favored hospital credentialing for 22 of the 40 procedures. CONCLUSIONS: General internists do a wide variety of procedures, the number and frequency of which are affected by the characteristics and location of their practice. Despite opinion to the contrary, more recently trained general internists do a wider variety of procedures than older colleagues. These results provide data to help define standards for training and credentialing and suggest that we should reexamine general internists' training in procedural skills.

Procedural skills training in internal medicine residencies. A survey of program directors.

STUDY OBJECTIVE: To obtain the opinions of internal medicine residency program directors about which procedural skills residents master during training and the amount of training needed to attain and maintain competence in each procedure. DESIGN: A mailed survey to all program directors in the United States. RESPONDENTS: Program directors or their designees from 389 of 431 (90%) internal medicine residency programs. RESULTS: For several procedures, 40% more respondents said all residents should master the procedure than said all their residents do master the procedure. Some procedures commonly done in practice were perceived as mastered by all residents in fewer than half of the programs. There were few differences in procedures learned by size or type of program. A fellowship program did affect exposure to some procedures in the field covered by the program. Median recommendations of training needed to master each procedure were similar to those of practicing internists for most procedures. CONCLUSIONS: Current residency training does not assure competency in all of the procedures the general internist does in practice. Program directors should examine which skills are adequately taught, test competence, and ways to improve residents' skills. Practicing general internists should have access to supervised training in procedural skills.

Ratings of residents' clinical competence and performance on certification examination.

The American Board of Internal Medicine (ABIM) requires directors of internal medicine residency programs to rate their residents in overall clinical competence and its essential components. In the study reported here, the authors investigated the relationships among these ratings and compared them with the residents' performance on the ABIM's certification examination in the years 1980 through 1985. The ratings of the individual components of clinical competence were correlated moderately with examination performance and very highly with each other. The individual components that were less dependent on medical knowledge (for example, interpersonal skills or humanism) had slightly smaller correlations with examination performance. The ratings of overall clinical competence were also related moderately to examination performance. Changes in the pattern of the ratings over time indicated that fewer candidates were receiving lower ratings while more were receiving higher ratings. The pass rates for each rating level were the same or lower; for example, the pass rate for candidates rated 5 was 68 percent in 1980 and 56 percent in 1985. The similar ranking of examinees by the program directors and the examination provides evidence for the validity of the examination.

A survey of clinical skills evaluation practices in internal medicine residency programs.

For the past 12 years, directors of residency training programs in internal medicine have had the responsibility for evaluating and reporting on the clinical competence of their residents who pursue certification by the American Board of Internal Medicine ( ABIM ). The board requires that residents demonstrate satisfactory clinical skills to take its certifying examination. In this article, the authors review the evaluation processes of 75 internal medicine residencies visited by the board in 1978-1982. The methods of evaluation used by the residencies are described and compared with the findings from an earlier cycle of visits in 1972-1975. In addition, information is presented on residents persuaded to leave the field of internal medicine. The results are encouraging, since they signify increased attention to the clinical method by graduate medical educators. This has been demonstrated by an improved, continuing commitment to the assessment of residents' clinical skills by program directors since the inception of the ABIM 's Hospital Visit Program.