McMaster at 50: lessons learned from five decades of PBL.

Although educators frequently act as if curricula are as standardized as drug doses (300 mg of PBL t.i.d.), such is not the case. As a case in point, at its inception, Problem Based Learning was hailed as a major curriculum innovation, with the promise of enormous gains in learning outcomes. Very quickly, ecclesiastical debates arose as what was true PBL and what was "modified PBL". Ironically, systematic reviews conducted fairly early in its evolution showed that the gains in learning outcome from PBL were neither large nor uniform (Vernon and Blake in Acad Med 68:550-563, 1993), and the most consistent finding was greater student satisfaction. In this paper, we review five decades of experience with the first PBL curriculum at McMaster. We point out how the curriculum has evolved, both theoretically and practically, in response to external influences, based both on empirical evidence and practical demands. We describe these changes in four broad domains-theoretical rationale, the curriculum, assessment and admissions.

Contexts, concepts and cognition: principles for the transfer of basic science knowledge.

CONTEXT: Transfer of basic science aids novices in the development of clinical reasoning. The literature suggests that although transfer is often difficult for novices, it can be optimised by two complementary strategies: (i) focusing learners on conceptual knowledge of basic science or (ii) exposing learners to multiple contexts in which the basic science concepts may apply. The relative efficacy of each strategy as well as the mechanisms that facilitate transfer are unknown. In two sequential experiments, we compared both strategies and explored mechanistic changes in how learners address new transfer problems. METHODS: Experiment 1 was a 2 × 3 design in which participants were randomised to learn three physiology concepts with or without emphasis on the conceptual structure of basic science via illustrative analogies and by means of one, two or three contexts during practice (operationalised as organ systems). Transfer of these concepts to explain pathologies in familiar organ systems (near transfer) and unfamiliar organ systems (far transfer) was evaluated during immediate and delayed testing. Experiment 2 examined whether exposure to conceptual analogies and multiple contexts changed how learners classified new problems. RESULTS: Experiment 1 showed that increasing context variation significantly improved far transfer performance but there was no difference between two and three contexts during practice. Similarly, the increased conceptual analogies led to higher performance for far transfer. Both interventions had independent but additive effects on overall performance. Experiment 2 showed that such analogies and context variation caused learners to shift to using structural characteristics to classify new problems even when there was superficial similarity to previous examples. CONCLUSIONS: Understanding problems based on conceptual structural characteristics is necessary for successful transfer. Transfer of basic science can be optimised by using multiple strategies that collectively emphasise conceptual structure. This means teaching must focus on conserved basic science knowledge and de-emphasise superficial features.

The mediating effect of context variation in mixed practice for transfer of basic science.

Applying a previously learned concept to a novel problem is an important but difficult process called transfer. Practicing multiple concepts together (mixed practice mode) has been shown superior to practicing concepts separately (blocked practice mode) for transfer. This study examined the effect of single and multiple practice contexts for both mixed and blocked practice modalities on transfer performance. We looked at performance on near transfer (familiar contexts) cases and far transfer (unfamiliar contexts) cases. First year psychology students (n = 42) learned three physiological concepts in a 2 × 2 factorial study (one or two practice contexts and blocked or mixed practice). Each concept was practiced with two clinical cases; practice context was defined as the number of organ systems used (one system per concept vs. two systems). In blocked practice, two practice cases followed each concept; in mixed practice, students learned all concepts before seeing six practice cases. Transfer testing consisted of correctly classifying and explaining 15 clinical cases involving near and far transfer. The outcome was ratings of quality of explanations on a 0-3 scale. The repeated measures analysis showed a significant near versus far by organ system interaction [F(1,38) = 3.4, p < 0.002] with practice with a single context showing lower far transfer scores than near transfer [0.58 (0.37)-0.83 (0.37)] compared to the two contexts which had similar far and near transfer scores [1.19 (0.50)-1.01 (0.38)]. Practicing with two organ contexts had a significant benefit for far transfer regardless of mixed or blocked practice; the single context mixed practice group had the lowest far transfer performance; this was a large effect size (Cohen's d = 0.81). Using only one practice context during practice significantly lowers performance even with the usually superior mixed practice mode. Novices should be exposed to multiple contexts and mixed practice to facilitate transfer.

How good is good? Students and assessors' perceptions of qualitative markers of performance.

BACKGROUND: Qualitative markers of performance are routinely used for medical student assessment, though the extent to which such markers can be readily translated to actionable pieces of information remains uncertain. PURPOSE: To explore (a) the perceived value to be indicated by descriptor phrases commonly used for describing student performance, (b) the perceived weight of the different performance domains (e.g. communication skills, work ethic, knowledge base, etc), and (c) whether or not the perceived value of the descriptors changes as a function of the performance domains. METHODS: Five domains of performance were identified from the thematic coding of past medical student transcripts (N = 156). From the transcripts, 91 distinct descriptors indicating the language commonly used by assessors were also identified. From the list of 91 descriptors, Thurstone's method of equal-appearing intervals was used to extract 10 descriptors that were representative of the continuum of student performance. A modified paired comparisons method was then used to enable the relative ranking of each of 10 descriptors combined with each of 5 different domains of performance. A web-based survey was used to collect responses from participants (N = 209), which consisted of medical students and faculty members who were previously involved in student assessment. RESULTS: Results demonstrated that respondents did not simply sum positive and negative descriptors in a uniform manner. Rather, comments on some domains (e.g., "ability to apply patient centred medicine") were seen as particularly positive when associated with positive descriptors but not particularly negative when associated with negative descriptors. For others (e.g., "receptivity and responsiveness to feedback") the reverse was true. Comments on "knowledge-base" elicited a relatively muted perception at both ends of the scale. Finally, the results also revealed moderate misalignment in the perceptions of assessors and students. CONCLUSIONS: The findings from this study suggest that the use of any given descriptor conveys slightly different meaning dependent on the context in which it is used. This helps to address some key issues surrounding the application of qualitative markers to performance assessment in medical education.

Integration and timing of basic and clinical sciences education.

BACKGROUND: Medical education has traditionally been compartmentalized into basic and clinical sciences, with the latter being viewed as the skillful application of the former. Over time, the relevance of basic sciences has become defined by their role in supporting clinical problem solving rather than being, of themselves, a defining knowledge base of physicians. METHODS: As part of the national Future of Medical Education in Canada (FMEC MD) project, a comprehensive empirical environmental scan identified the timing and integration of basic sciences as a key pressing issue for medical education. Using the literature review, key informant interviews, stakeholder meetings, and subsequent consultation forums from the FMEC project, this paper details the empirical basis for focusing on the role of basic science, the evidentiary foundations for current practices, and the implications for medical education. FINDINGS: Despite a dearth of definitive relevant studies, opinions about how best to integrate the sciences remain strong. Resource allocation, political power, educational philosophy, and the shift from a knowledge-based to a problem-solving profession all influence the debate. There was little disagreement that both sciences are important, that many traditional models emphasized deep understanding of limited basic science disciplines at the expense of other relevant content such as social sciences, or that teaching the sciences contemporaneously rather than sequentially has theoretical and practical merit. Innovations in integrated curriculum design have occurred internationally. Less clear are the appropriate balance of the sciences, the best integration model, and solutions to the political and practical challenges of integrated curricula. DISCUSSION: New curricula tend to emphasize integration, development of more diverse physician competencies, and preparation of physicians to adapt to evolving technology and patients' expectations. Refocusing the basic/clinical dichotomy to a foundational/applied model may yield benefits in training widely competent future physicians.

The effect of conceptual and contextual familiarity on transfer performance.

Applying a previously learned concept to a novel problem is an important but difficult process called transfer. It is suggested that a commonsense analogy aids in transfer by linking novel concepts to familiar ones. How the context of practice affects transfer when learning using analogies is still unclear. This study investigated the effect of a commonsense analogy and context familiarity for transfer of physiological concepts. First year psychology students (n = 24) learned three concepts: Starling's law, Laplace's law, and laminar-turbulent flow. The control group saw standard explanations while the intervention group saw an additional commonsense analogy. The context of learning was the organ system used for two practice clinical cases which differed for all concepts. Testing consisted of 12 new clinical cases. Starling's law cases used the organ system from practice while the other concepts presented in both novel and familiar organ systems. Half of the sample repeated testing after 1 week delay. The outcome was ratings of explanations of cases on a 0-3 scale. The effect of analogy was significant (Mean = 1.24 with, 0.86 without, F(1,22) = 4.26, p < 0.05) but not after delay (means of 1.08 and 0.75 respectively, F = (1,10), p = 0.06) There was significant effect for familiar context (Same = 1.23 (Starling), different = 0.68 (Laplace) and 0.73 (laminar-turbulent flow) (F(2,44) = 5.14, p < 0.01). Laplace's law and laminar turbulent flow cases in the familiar organ system had means of 1.65 and 1.77 respectively compared to novel cases with means of 0.74 and 0.68 (F(1,22) = 35.64, p < 0.0001). Similar effects were observed after delay. There was significant decay in performance after delay for all participants (immediate = 1.17, delayed = 0.91, F = 11.9 (1,10) p < 0.01). Common analogies aid conceptual understanding necessary for transfer. Despite conceptual aids, solving transfer problems is difficult.

Assessment steers learning down the right road: impact of progress testing on licensing examination performance.

Although it is generally accepted that assessment steers learning, this is generally viewed as an undesirable side effect. Recent evidence suggests otherwise. Experimental studies have shown that periodic formative assessments can enhance learning over equivalent time spent in study (Roediger & Karpicke 2006). However, positive effects of assessment at a curriculum level have not been demonstrated. Progress tests are a periodic formative assessment designed to enhance learning by providing objective and cumulative feedback, and by identifying a subgroup of students who require additional remediation. McMaster adopted the progress test methods in 1992-1993, as a consequence of poor performance on a national licensing examination. This article shows the positive effect of this innovation, which amounts to an immediate increase of about one-half standard deviation in examination scores, and a consistent upward trend in performance. The immediate effect of introducing objective tests was a reduction in failure rate on the licensing examination from 19% to 4.5%. Various reasons for this improvement in performance are discussed.

The supportive care needs of family members of men with advanced prostate cancer.

This exploratory study identified the supportive care needs of family members of men with advanced hormone-sensitive (HS) and hormone-refractory (HR) prostate cancer. In focus groups and individual interviews, we asked eight family members of men with HS disease and 11 family members of men with HR disease to identify their supportive care needs and recommend strategies for improving care to meet these needs. Unmet needs common to both groups were lack of information and uncertainty about the future. Unmet needs specific to family members affected by HR prostate cancer related to caregiver burden, practical assistance, and isolation. Implications for practice to improve supportive care services for families affected by APC are provided.

Evaluating health-related quality of life and priority health problems in patients with prostate cancer: a strategy for defining the role of the advanced practice nurse.

A framework for the introduction and evaluation of APN roles emphasizes the importance of a systematic approach to role development based on the assessment of patient health needs. This study determined the health-related quality of life (HRQL) of patients with prostate cancer. The most frequent and severe patient health problems and their perceptions of priority health problems were identified and compared across five patient groups as a strategy to inform the supportive care role of the advanced oncology nurse for patients with advanced prostate cancer. The study found that the majority of men with early stage and advanced hormone sensitive prostate cancer can expect to enjoy good quality of life for several years following diagnosis. These two patient groups have common priority needs for improving their health related to sexual function, urinary frequency, urinary incontinence, and physical activity. Both groups may benefit from an advanced practice nursing (APN) role that can provide episodic supportive care for health problems occurring at different treatment stages. Conversely, it was found that men with advanced hormone refractory prostate cancer experience significantly poorer HRQL and have multiple severe health problems. These patients also have different priority needs including problems related to pain, fatigue, and decreased physical activity. Because of this, the focus of supportive care programs and interventions in advanced prostate cancer will differ for those with hormone refractory disease. They may benefit more from an APN role that can provide ongoing rather than episodic supportive care to assess and manage the multiple, new, and worsening health problems associated with progressive disease.

Outcomes of trimodality bladder-sparing therapy for muscle-invasive bladder cancer.

INTRODUCTION: Although radical cystectomy is considered the standard of care for muscle-invasive bladder cancer (MIBC), recent data has suggested comparable survival outcomes for bladder-sparing trimodality therapy (TMT). We conducted a retrospective, single-institution analysis of MIBC patients to evaluate the efficacy of TMT as an alternative, curative approach to surgical intervention. METHODS: We conducted a retrospective analysis of MIBC patients assessed by a multidisciplinary team at the Juravinski Cancer Centre from 2010-2016. Patients underwent transurethral resection of bladder tumor (TURBT) followed by radiotherapy with or without concurrent chemotherapy. Patients could receive neoadjuvant treatment. Clinical data and response rates were summarized, and overall survival (OS) and disease-free survival (DFS) were estimated using the Kaplan-Meier method. RESULTS: Our analytic cohort included 115 patients, of whom 53 underwent TMT and 62 underwent radiotherapy alone following TURBT. Median age at diagnosis was 79 years and median followup was 21 months. Complete response rates in those receiving TMT and radiation without chemotherapy were 84.4% and 66.7%, respectively. For TMT patients, three-year OS and DFS were 68.5% and 49.6%, respectively. Patients who received TMT had reduction in risk of mortality (hazard ratio [HR] 0.49; p=0.026) and disease recurrence (HR 0.55; p=0.017) compared to those who had radiation without chemotherapy. Overall, four patients had grade 3 or higher late toxicity. CONCLUSIONS: In this single-institution analysis, TMT appears to be a safe and effective approach in the short-term management of MIBC in appropriately selected patients. Extended followup and analysis are necessary to validate these results.

Problem-based learning and medical education forty years on. A review of its effects on knowledge and clinical performance.

Problem-based learning (PBL) has swept the world of medical education since its introduction 40 years ago, leaving a trail of unanswered or partially answered questions about its benefits. The literature is replete with systematic reviews and meta-analyses, all of which have identified some common themes; however, heterogeneity in the definition of a 'problem-based learning curriculum' and its delivery, coupled with different outcome measurements, has produced divergent opinions. Proponents and detractors continue to dispute the merits of the cognitive foundation of a PBL approach, but, despite this, there is evidence that graduates of PBL curricula demonstrate equivalent or superior professional competencies compared with graduates of more traditional curricula.

PBL in the undergraduate MD program at McMaster University: three iterations in three decades.

When the undergraduate MD program of McMaster University admitted its first cohort of 20 students in 1969, it heralded a major change in medical school pedagogy that has influenced the education of medical students around the world. The three-year PBL curriculum, which emphasized small-group tutorials, self-directed learning, a minimal number of didactic presentations, and student evaluation that was based almost entirely on performance in the tutorial, represented a radical departure from traditional curricula. Since the inception of the original curriculum in 1969, there have been two major curriculum revisions, the most recent of which was in 2005. The original curriculum attempted to integrate both basic science and clinical science into the biomedical problems. The second iteration of the curriculum focused on priority health problems and centered on a list of common medical problems as the foundation for curriculum organization, on the basis that an understanding of the management of common conditions included areas of knowledge that would be essential for clinical competence. Under the third, current curriculum, the COMPASS (concept-oriented, multidisciplinary, problem-based, practice for transfer, simulations in clerkship, streaming) model was adopted. Under this concept-based system, emphasis is placed on underscoring the underlying concepts in the curriculum with a logical sequencing of both the concepts and the body systems. This article briefly reviews the history of the development of the undergraduate MD program at McMaster and the three curricula that have been developed during the past three decades.

The value of basic science in clinical diagnosis.

BACKGROUND: The role of basic science knowledge in clinical diagnosis is unclear. There has been no experimental demonstration of its value in helping students recall and organize clinical information. This study examines how causal knowledge may lead to better recall and diagnostic skill over time. METHOD: Undergraduate medical students learned either four neurological or rheumatic disorders. One group learned a basic science explanation for the symptoms. The other learned epidemiological information. Both were then tested with the same set of clinical cases immediately after learning and one week later. RESULTS: On immediate test, there was no difference in accuracy (70% for both groups). However, one week later, performance in the epidemiology group dropped to 51%; the basic science group only dropped to 62%. CONCLUSIONS: Basic science knowledge relating causal knowledge to disease symptoms can improve diagnostic accuracy after a delay.

The Impact of Emotion on Learners' Application of Basic Science Principles to Novel Problems.

PURPOSE: Training to become a physician is an emotionally laden experience. Research in cognitive psychology indicates that emotions can influence learning and performance, but the materials used in such research (e.g., word lists) rarely reflect the complexity of material presented in medical school. The present study examined whether emotions influence learning of basic science principles. METHOD: Fifty-five undergraduate psychology students were randomly assigned to write about positive, negative, or neutral life events for nine minutes. Participants were then taught three physiological concepts, each in the context of a single organ system. Testing consisted of 13 clinical cases, 7 presented with the same concept/organ system pairing used during training ("near transfer") and 6 with novel pairings ("far transfer"). Testing was repeated after one week with 13 additional cases. RESULTS: Forty-nine students provided complete data. Higher test scores were found when the concept/organ system pairing was held constant (near transfer = 51% correct vs. far = 33%; P < .001). Emotion condition influenced participants' overall performance, with individuals in the neutral condition (50.1%) performing better than those in the positive (38.2%, P < .05) and negative (37.7%, P < .001) emotion conditions. CONCLUSIONS: These data suggest that regardless of whether the emotion is positive or negative, mild affective states can impair learning of basic science concepts by novices. Demands on working memory and subsequent cognitive load provide a potential explanation. Future work will examine the extent to which these findings generalize to medical trainees.

Medline for Medical Students? Searching for the Right Answer.

Objective: Does MEDLINE use, when added to more traditional sources of information, improve the accuracy of medical students' clinical decisions when compared to those obtained using traditional sources only? Design: Randomized control trial. Setting: McMaster University Faculty of Medicine, Undergraduate Program. Participants: The entire class of 101 medical students, class of 1998. Overall response rate on 9 items was 56% (510/909), with 35% (35/101) completing all 9 items. Intervention: All participants were randomized on each of the nine clincal scenarios for which the student could choose to apply, or refrain from applying, a proferred intervention. When randomized to the control arm, the student used traditional sources of information for decision-making. When randomized to the experimental arm, the student used MEDLINE searching in addition to more traditional sources of information, for decision-making. Main Outcome Measures: Prior to, and subsequent to the information search, the students indicated their comfort in using the proffered intervention on a seven point Likert scale. Results: Analyzed with one-way ANOVA, the mean rating post-search of the control non- MEDLINE arm was 2.94 (SD = 1.80) (where 1 = correct, 7 = incorrect) and of the experimental MEDLINE arm was 2.71 (SD = 1.81), (F(1,522) = 2.03, p = 0.15 n.s). The mean change of the control arm was 0.97 (SD = 2.04) and of the experimental arm was 1.008 (SD = 1.92), (F(1,511) = 0.04, p = 0.84n.s.). Conclusions: The addition of MEDLINE to more traditional answer-seeking behaviors by medical students does not translate into a beneficial impact on clinical decision-making.

The use of triangulation in qualitative research.

Triangulation refers to the use of multiple methods or data sources in qualitative research to develop a comprehensive understanding of phenomena (Patton, 1999). Triangulation also has been viewed as a qualitative research strategy to test validity through the convergence of information from different sources. Denzin (1978) and Patton (1999) identified four types of triangulation: (a) method triangulation, (b) investigator triangulation, (c) theory triangulation, and (d) data source triangulation. The current article will present the four types of triangulation followed by a discussion of the use of focus groups (FGs) and in-depth individual (IDI) interviews as an example of data source triangulation in qualitative inquiry.

The supportive care needs of men with advanced prostate cancer.

PURPOSE/OBJECTIVES: To better understand the priority supportive care needs of men with advanced prostate cancer. RESEARCH APPROACH: Qualitative, descriptive study. SETTING: Outpatient cancer center and urology clinics in central western Ontario, Canada. PARTICIPANTS: 12 men with hormone-sensitive prostate cancer and 17 men with hormone-refractory prostate cancer. METHODOLOGIC APPROACH: Patients participated in focus groups and interviews that examined their supportive care needs, their priority needs, and suggestions for improvements to the delivery of care. Tape-recorded focus group discussions and interviews were organized using NVivo software. MAIN RESEARCH VARIABLES: Patients' supportive care needs. FINDINGS: Participants identified prostate cancer-specific information and support to maintain their ability "to do what they want to do" as priority needs. Both hormone-sensitive and hormone-refractory groups cited problems with urinary function, the side effects of treatment, fatigue, and sexual concerns as major functional issues. Participants experienced emotional distress related to diagnosis and treatment. CONCLUSIONS: A priority health need for men with advanced prostate cancer is to improve or maintain functional abilities. In addition, men require support to meet their stage-specific information needs and to address concerns about the diagnosis and ambivalent feelings about past treatment decisions. INTERPRETATION: Nurses could play an important role in addressing men's information needs and providing emotional support. The complex care needs of men with advanced prostate cancer provide opportunity for the development of advanced practice nurse roles that would use the clinical and nonclinical aspects of the role.