Strategies for sustaining quality in PBL facilitation for large student cohorts.

Problem-based learning (PBL) has been used to scaffold and support student learning in many Australian medical programs, with the role of the facilitator in the process considered crucial to the overall educational experience of students. With the increasing size of student cohorts and in an environment of financial constraint, it is important to develop quality control procedures to maintain an effective and efficient PBL program and a stable tutor workforce. This paper reports on organisational practices and faculty development opportunities used to sustain facilitation quality in a large cohort PBL program. Seven strategies are proposed, built around a sound professional development program, and a suite of tactics for recruitment of, and ongoing support for, PBL tutors.

A review of longitudinal community and hospital placements in medical education: BEME Guide No. 26.

BACKGROUND: Traditionally, clinical learning for medical students consists of short-term and opportunistic encounters with primarily acute-care patients, supervised by an array of clinician preceptors. In response to educational concerns, some medical schools have developed longitudinal placements rather than short-term rotations. Many of these longitudinal placements are also integrated across the core clinical disciplines, are commonly termed longitudinal integrated clerkships (LICs) and often situated in rural locations. This review aimed to explore, analyse and synthesise evidence relating to the effectiveness of longitudinal placements, for medical students in particular to determine which aspects are most critical to successful outcomes. METHOD: Extensive search of the literature resulted in 1679 papers and abstracts being considered, with 53 papers ultimately being included for review. The review group coded these 53 papers according to standard BEME review guidelines. Specific information extracted included: data relating to effectiveness, the location of the study, number of students involved, format, length and description of placement, the learning outcomes, research design, the impact level for evaluation and the main evaluation methods and findings. We applied a realist approach to consider what works well for whom and under what circumstances. FINDINGS: The early LICs were all community-based immersion programs, situated in general practice and predominantly in rural settings. More recent LIC innovations were situated in tertiary-level specialist ambulatory care in urban settings. Not all placements were integrated across medical disciplines but were longitudinal in relation to location, patient base and/or supervision. Twenty-four papers focussed on one of four programs from different viewpoints. Most evaluations were student opinion (survey, interview, focus group) and/or student assessment results. Placements varied from one half day per week for six months through to full time immersion for more than 12 months. The predominant mechanism relating to factors influencing effectiveness was continuity of one or more of: patient care, supervision and mentorship, peer group and location. The success of LICs and participation satisfaction depended on the preparation of both students and clinical supervisors, and the level of support each received from their academic institutions. CONCLUSION: Longitudinal placements, including longitudinal integrated placements, are gaining in popularity as an alternative to traditional block rotations. Although relatively few established LICs currently exist, medical schools may look for ways to incorporate some of the principles of LICs more generally in their clinical education programmes. Further research is required to ascertain the optimum length of time for placements depending on the defined learning outcomes and timing within the programme, which students are most likely to benefit and the effects of context such as location and type of integration.

Perceptions of job satisfaction relating to affective organisation commitment.

OBJECTIVES: Affective organisation commitment, which refers to a psychological attachment to, and involvement with, an employing institution, is regarded as important because of its effects on employee identification with the employer and its causal effects on work effort and staff retention. This paper explores the experiences of casual tutors facilitating problem-based learning (PBL) tutorials and aims to identify aspects of their role that strengthen and detract from employee job satisfaction and affective commitment. METHODS: Qualitative data were gathered from first- and second-year tutors (N = 13) through 2 focus groups. Both clinicians and non-clinicians were recruited, including highly experienced staff and those with < 12 months of tutoring experience. RESULTS: Four main themes arose from inductive analysis of data: job-related factors; job-involvement characteristics; professional challenges and responsibilities, and mentoring for learning and support. The first 2 themes are congruent with previous literature on organisation commitment; novel findings include the supportive and compensatory nature of the collegial relationships formed between casual tutors. Role attenuation, a job-related factor, was a predominant perception as it related to dysfunctional groups and increasing student disengagement with PBL. CONCLUSIONS: Within the unique learning environment of PBL, positive factors relating to job satisfaction may have an important role to play in improving tutors' commitment to their employing organisation. Aspects of the role which are viewed most negatively and relate most significantly to affective commitment need to be addressed promptly. Attention should be directed to supporting tutors to maximise the perceived benefits and providing professional development and improved communication to better address issues associated with difficult or disengaged students as well as isolation from decision-makers.

Students generating questions for their own written examinations.

Assessment partnerships between staff and students are considered a vital component of the student-centred educational process. To enhance the development of this partnership in a problem-based learning curriculum, all first-year students were involved in generating a bank of formative assessment questions with answers, some of which were included in their final written examination. Important principles to guide development of a sound methodology for such an assessment partnership have been described. These include organisational issues as well as matters pertaining to participation, education and motivation of students and teaching staff.

An exploration of perceptions of tutor evaluation in problem-based learning tutorials.

CONTEXT: Within problem-based learning (PBL) tutorials, the relationship between student and tutor is predicated on the tutor adopting the role of mentor and metacognitive coach. This rapport differs considerably from the traditional teacher-student relationship and is likely to impact on the process and outcomes of tutor evaluations. Such evaluations are a ubiquitous means of providing feedback to tutors from students about the quality of their facilitation. Although critiqued in the literature as 'popularity contests', tutor evaluations are commonly used in tertiary institutions for purposes of recruitment, re-employment and promotion. METHODS: This study seeks to provide insight into students' and tutors' perceptions of evaluations of teaching within PBL tutorials. As a unique teaching and learning environment, the PBL tutorial requires sophisticated facilitation skills of tutors and considerable autonomy from students. Qualitative data were gathered from three focus group discussions and one in-depth interview with first- and second-year medical students and their PBL tutors. RESULTS: Thematic analysis identified four major themes, including: defining the 'ideal' tutor; making unthinking or deliberately false evaluations; promoting a consumer mentality, and providing support for tutors. An underlying suspicion of the purpose of the evaluation process was apparent among tutors and students. CONCLUSIONS: Findings suggest that, within the PBL tutorial environment at least, regularly evaluating tutors creates mistrust and confusion among the medical school, the tutor and the student on several levels. Suggestions for further research are proposed.

Conducting the symphony: a qualitative study of facilitation in problem-based learning tutorials.

CONTEXT: Tutors in problem-based learning (PBL) tutorials have a complex role to play in facilitating students' learning. This includes providing support for students' acquisition of content knowledge and skills in critical thinking, coaching of group processes and modelling of reflective practice. Few studies which investigate the key role of tutors in the PBL tutorial process are qualitative in design. METHODS: This study explores the nature and technique of facilitation provided by PBL tutors from the students' viewpoint. Data were obtained from written responses to an open-ended question asking students about the effectiveness of their PBL tutor(s) and from in-depth interviews carried out with two randomly selected students. RESULTS: Three main themes arose from the inductive analysis of qualitative data: (i) role confusion by tutors; (ii) tutor management of sensitive issues, and (iii) facilitation 'style'. The theme of tutors' facilitation style was dominant and three sub-categories were apparent. These were: (i) managing the learning in PBL tutorials; (ii) facilitating group processes, and (iii) guiding group discussion. CONCLUSIONS: Findings highlight the need for tutors to regularly review the PBL tutorial processes and group dynamics within the tutorial setting. These findings have implications for tutor training and programmes of ongoing professional development for PBL facilitators.

Are deep strategic learners better suited to PBL? A preliminary study.

The aim of this study was to determine if medical students categorized as having deep and strategic approaches to their learning find problem-based learning (PBL) enjoyable and supportive of their learning, and achieve well in the first-year course. Quantitative and qualitative data were gathered from first-year medical students (N = 213). All students completed the Medical Course Learning Questionnaire at the commencement and completion of their first year of medical studies. The instrument measured a number of different aspects of learning, including approaches to learning, preferences for different learning environments, self-efficacy, and perceptions of learning within PBL tutorials. Qualitative data were collected from written responses to open questions. Results of students' performance on two forms of examinations were obtained for those giving permission (N = 68). Two-step cluster analysis of the cohort's responses to questions about their learning approaches identified five clusters, three of which represented coherent combinations of learning approaches (deep, deep and strategic, and surface apathetic) and two clusters which had unusual or dissonant combinations. Deep, strategic learners represented 25.8% of the cohort. They were more efficacious, preferred learning environments which support development of understanding and achieved significantly higher scores on the written examination. Strongly positive comments about learning in PBL tutorials were principally described by members of this cluster. This preliminary study employed a technique to categorize a student cohort into subgroups on the basis of their approaches to learning. One, the deep and strategic learners, appeared to be less vulnerable to the stresses of PBS in a medical course. While variation between individual learners will always be considerable, this analysis has enabled classification of a student group that may be less likely to find PBL problematic. Implications for practice and suggestions for future research are proposed.

An analysis of peer, self, and tutor assessment in problem-based learning tutorials.

OBJECTIVE: The purpose of this study was to explore self-, peer-, and tutor assessment of performance in tutorials among first year medical students in a problem-based learning curriculum. METHODS: One hundred and twenty-five students enrolled in the first year of the Bachelor of Medicine and Bachelor of Surgery Program at the University of Queensland were recruited to participate in a study of metacognition and peer- and self-assessment. Both quantitative and qualitative data were collected from the assessment of PBL performance within the tutorial setting, which included elements such as responsibility and respect, communication, and critical analysis through presentation of a case summary. Self-, peer-, and tutor assessment took place concurrently. RESULTS: Scores obtained from tutor assessment correlated poorly with self-assessment ratings (r = 0.31-0.41), with students consistently under-marking their own performance to a substantial degree. Students with greater self-efficacy, scored their PBL performance more highly. Peer-assessment was a slightly more accurate measure, with peer-averaged scores correlating moderately with tutor ratings initially (r = 0.40) and improving over time (r = 0.60). Students consistently over-marked their peers, particularly those with sceptical attitudes to the peer-assessment process. Peer over-marking led to less divergence from the tutor scoring than under-marking of one's own work. CONCLUSION: According to the results of this study, first-year medical students in a problem-based learning curriculum were better able to accurately judge the performance of their peers compared to their own performance. This study has shown that self-assessment of process is not an accurate measure, in line with the majority of research in this domain. Nevertheless, it has an important role to play in supporting the development of skills in reflection and self-awareness.

Effects of a metacognitive intervention on students' approaches to learning and self-efficacy in a first year medical course.

AIM: To determine the influence of metacognitive activities within the PBL tutorial environment on the development of deep learning approach, reduction in surface approach, and enhancement of individual learning self-efficacy. METHOD: Participants were first-year medical students (N = 213). A pre-test, post-test design was implemented with intervention and control cohorts, with intervention students experiencing a program of metacognitive activities within their PBL tutorials of at least 20 weeks duration. All students completed the Medical Course Learning Questionnaire at the commencement, and again at the completion of, the study. The metacognitive intervention itself consisted of reflection on the learning in PBL coupled with peer- and self-assessment. RESULTS: Self-efficacy was significantly reduced for both control and intervention cohorts at the conclusion of the study. A significant reduction in the adoption of deep and strategic learning approach, matched by a corresponding increase in the use of surface learning, was demonstrated for both cohorts. There was a statistically significant association between high self-efficacy and deep learning approach, with older students over-represented in the group of efficacious deep learners. CONCLUSION: Over the course of first-year medical studies, students lose self-efficacy and move away from deep-strategic learning approaches towards more surface approaches. The program of metacognitive activities failed to reverse this trend. The substantial swing towards surface learning raises questions about the perceived capacity of PBL curricula to promote deep approaches to learning in dense curricula, and reinforces the importance of personal and contextual factors, such as study habits, workload and assessment, in determining individual approaches and idiosyncratic responses to learning situations.

Peer assessment in problem-based learning: a qualitative study.

Peer assessment provides a powerful avenue for students to receive feedback on their learning. Although student perceptions of peer assessment have been studied extensively in higher education, little qualitative research has been undertaken with medical students in problem-based learning (PBL) curricula. A qualitative study of students' attitudes to, and perceptions of, peer assessment was undertaken within the framework of a larger study of metacognition with first-year medical students at the University of Queensland. A highly structured format for provision of feedback was utilised in the study design. Many recommendations from the higher education literature on optimal implementation of peer-assessment procedures were put into practice. Results indicated the existence of six main themes: (1) increased responsibility for others, (2) improved learning, (3) lack of relevancy, (4) challenges, (5) discomfort, and (6) effects on the PBL process. Five of these themes have previously been described in the literature. However, the final theme represents a unique, although not unexpected, finding. Students expressed serious concerns about the negative impact of peer assessment on the cooperative, non-judgmental atmosphere of PBL tutorial groups. The practical implications of these findings are considered.

Whispered voice test for screening for hearing impairment in adults and children: systematic review.

OBJECTIVE: To determine the accuracy of the whispered voice test in detecting hearing impairment in adults and children. DESIGN: Systematic review of studies of test accuracy. DATA SOURCES: Medline, Embase, Science Citation Index, unpublished theses, manual searching of bibliographies of known primary and review articles, and contact with authors. STUDY SELECTION: Two reviewers independently selected and extracted data on study characteristics, quality, and accuracy of studies. Studies were included if they had cross sectional designs, at least one of the index tests was the whispered voice test, and the reference test (audiometry) was performed on at least 80% of the participants. DATA EXTRACTION: Data were used to form 2x2 contingency tables with hearing impairment by audiometry as the reference standard. DATA SYNTHESIS: The eight studies that were found used six different techniques. The sensitivity in the four adult studies was 90% or 100% and the specificity was 70% to 87%. The sensitivity in the four childhood studies ranged from 80% to 96% and specificity ranged from 90% to 98%. CONCLUSION: The whispered voice test is a simple and accurate test for detecting hearing impairment. There is some concern regarding the lower sensitivity in children and the overall reproducibility of the test, particularly in primary care settings. Further studies should be conducted in primary care settings to explore the influence of components of the testing procedure to optimise test sensitivity and to promote standardisation of the testing procedure.