Measuring instructional quality in community-orientated medical education: looking into the black box.

OBJECTIVES: Decentralizing medical education to community settings has raised issues of instructional quality. The need to evaluate community-based instruction accents the need to adopt a systems perspective, moving beyond factors known to comprise general clinical teaching effectiveness to include factors that focus on instruction as a process. Application of evaluation models using traditional input-output analysis can be flawed. This approach--dubbed the 'black box'--typically examines inputs and outputs, but often ignores throughputs. DESIGN: In this article we open the black box, using theory to examine the underlying processes that define community-based medical education. We first describe the components and processes of an instructional model that is framed by the philosophy of quality and grounded in experiential learning theory. Without examining the critical processes at work inside the black box - i.e. how students come to acquire clinical knowledge and how behaviours are influenced - it is difficult to assess which programme features contribute to success. Tensions created by the absence of consensus on the outcomes of instruction and the challenge of developing adequate measures are highlighted. SETTING: State University of New York at Buffalo. SUBJECTS: Clinician-teachers, learners and patients in the environment. RESULTS AND CONCLUSIONS: We conclude with describing a tool for evaluating community-based instruction that is guided by the context of our model.

Exploring instructional quality indicators in ambulatory medical settings: an ethnographic approach.

BACKGROUND AND OBJECTIVES: As medical education moves to community settings, the quality of learning is influenced by differences in the practice environment, organization, resources, patient case mix, and demographics. This ethnographic study identified experiences and processes that influence student learning in community-based practice settings. METHODS: Trained field researchers conducted participant observation in eight community teaching sites. Data were analyzed using a qualitative, grounded theory approach. RESULTS: Three dominant themes emerged: 1) the preceptor's role in situating learning opportunities, 2) the learner's role in transforming experience into learning, and 3) the practice organization as a classroom setting. The findings highlight the importance of exploiting learning opportunities and the contributions of other medical staff and patients in facilitating unique learning experiences. CONCLUSIONS: This research suggests the need to move beyond the typical student ratings of teacher effectiveness to consider and assess additional important factors and processes that affect instructional quality.

Moving from policy to practice in curriculum change at the University of Virginia School of Medicine, East Carolina University School of Medicine, and SUNY-Buffalo School of Medicine.

A set of formative evaluation studies from the medical schools of the University of Virginia (UVA), East Carolina University (ECU), and the State University of New York at Buffalo (SUNY-Buffalo) portrays, in qualitative and quantitative terms, evidence of achievements and obstacles to the curricular reform supported by The Robert Wood Johnson Foundation's Generalist Physician Initiative (GPI). In this paper, innovations in the under-graduate curriculum, a specific course, and instructional strategies are examined. Individual interviews of faculty and focus groups with students assessed opinions about curricular change at the University of Virginia. Questionnaires and focus groups provided information about the impact of course changes at East Carolina University. Questionnaires completed by students provided information of the effect of modifying the instructional strategies at SUNY-Buffalo. The obstacles to implementing change at the three schools included breakdowns in the faculty's understanding and support of change, lack of skills required to implement change, and weakness in coordinating and assessing planned change. Although the GPI catalyzed changes in the content and conduct of generalist education at the three schools, many lessons were learned that are applicable to other medical schools.