Assessing professionalism: a review of the literature.

Although assessing professionalism poses many challenges, gauging and detecting changes in professionalism is impossible without measurement. This paper is a review of techniques used to assess professionalism during the past 20 years. The authors searched five electronic databases and reference lists from 1982 to 2002. Eighty-eight assessments were retained and organized into content area addressed (i.e. ethics, personal characteristics, comprehensive professionalism, diversity) and type of outcome examined (i.e. affective, cognitive, behavioral, environmental). Instead of creating new professionalism assessments, existing assessments should be improved. Also, more studies on the predictive validity of assessments and their use as part of formative evaluation systems are recommended. Based on the review, suggestions are presented for assessing medical students, resident physicians and practicing physicians.

Assessing practice-based learning and improvement.

BACKGROUND: Practice-based learning and improvement (PBLI) is 1 of 6 general competencies expected of physicians who graduate from an accredited residency education program in the United States and is an anticipated requirement for those who wish to maintain certification by the member boards of the American Board of Medical Specialties. This article describes methods used to assess PBLI. SUMMARY: Six electronic databases were searched using several search terms pertaining to PBLI. The review indicated that 4 assessment methods have been used to assess some or all steps of PBLI: portfolios, projects, patient record and chart review, and performance ratings. Each method is described, examples of application are provided, and validity, reliability, and feasibility characteristics are discussed. CONCLUSION: Portfolios may be the most useful approach to assess residents' PBLI abilities. Active participation in peer-driven performance improvement initiatives may be a valuable approach to confirm practicing physician involvement in PBLI.

Descriptive meta-evaluation. Case study of an interdisciplinary curriculum.

This article describes an example of meta-evaluation in an educational setting. The meta-evaluation examined an evaluation of a community-based, interdisciplinary curriculum. The Program Evaluation Standards (PES), divided into the categories of utility, feasibility, propriety, and accuracy, provided a framework for the meta-evaluation. Utility standards address the information needs of intended users. Feasibility refers to the extent to which an evaluation is realistic, prudent, diplomatic, and frugal. Propriety refers to the properness of an evaluation in terms of meeting legal and ethical obligations. Accuracy pertains to the trustworthiness of evaluation data. Use of the PES as a framework for descriptive meta-evaluation of a single case illustrated the breadth of issues involved in curriculum evaluation and their interrelatedness. Furthermore, the PES helped to reveal strengths and weaknesses that served as starting points for further improvement of the evaluation.

Family practice graduate preparedness in the six ACGME competency areas: prequel.

BACKGROUND AND OBJECTIVES: Since July 2002, family practice residency program accreditation requires evidence of teaching and assessing residents in six competency areas. This study was conducted to obtain baseline information about family practice graduates' perceptions of the importance of specific competencies and the extent to which residency training prepared them to perform skills representative of the six competency areas. METHODS: A national, cross-sectional survey was conducted of family physicians who had graduated from residency programs from 1998 to 2000. RESULTS: The response rate was 54% (n=1,228). Graduates reported the most preparation in patient care skills, followed by interpersonal and communication skills and then professionalism. The least preparation was reported for skills pertinent to practice-based learning and improvement, systems-based practice, and some areas of professionalism. CONCLUSIONS: Areas of residency education that appear to warrant improvement include education about system aspects of care, practice-based learning and improvement, and selected professionalism issues.

Professionalism: identifying current themes.

PURPOSE OF REVIEW: This review analyzes the literature on medical professionalism in order to inform further study, educational activity, and reflective practice for all phases of a physician's professional development from medical school through practice. RECENT FINDINGS: Several themes emerged from an analysis of the writing about medical professionalism during the past year. A number of authors attempted to identify concrete behaviors associated with attributes and characteristics used to define professionalism. These behaviors in turn became the focus of teaching and assessment activities primarily conducted with medical students and residents. Educators who attempted to assess professionalism achieved some modest success in reporting valid and reliable results of their efforts. Each of these activities points to a systemic component of professionalism that includes five broad categories of relationships in which physicians engage. All five categories are important to understanding and acting on the values and attitudes required by professionalism in medicine. SUMMARY: Competence to practice medicine includes the ability of physicians to demonstrate professionalism in all the relationships in which they engage. The attributes and characteristics used to define professionalism contribute to recognizing the behaviors that should be apparent not only in the physician-to-patient relationship which is at its core, but also in relationships with other physicians, colleagues in the health care system, society, and oneself. All these relationships must be appropriately aligned with the values and attitudes that form a collective understanding of professionalism that has emerged within the profession.

Chaos theory as a planning tool for community-based educational experiences for health students.

Educational and community health systems are social systems composed of a group or collection of entities for which there is a unifying principle. The purpose of this paper is to briefly explain chaos theory and to apply it to the Interdisciplinary Rural Health Training Program (IRHTP) as a case study. The IRHTP is an existing rural, community based educational program for baccalaureate and graduate health care students. Chaos theory attempts to understand the underlying order in processes that appear to not have any guidelines or principles. These processes typically involve the interaction of several elements over time. Chaos theory provided the university with a method of anticipating the natural flux between order and chaos to allow the system to function at its highest level. To thrive in such a complex dynamic environment the authors recommend application of Ockerman's Five Factors.

A Rationale for Using Synthetic Designs in Medical Education Research.

The extent to which the results of a study can be attributed to the intervention under investigation (i.e., internal validity) is an important consideration in interpreting study findings. There are many threats to the internal validity of designs frequently used in medical education research. Synthetic designs, which involve the integration of two or more weak designs, or the addition of design elements, may afford investigators greater control over confounding variables in medical education research. A rationale for using synthetic designs is presented and two examples of their use in medical education settings are examined. The concluding proposition is that synthetic designs allow investigators flexibility in planning research that is feasible in medical education settings. In addition, they may permit stronger causal inferences between interventions and results than traditional research designs.