Evaluative judgement in practice education: How does the ability to judge the quality of work impact placement performance?

INTRODUCTION: Occupational therapy students need to be ready to work autonomously in a range of environments as soon as they complete their degree. Practice education experiences are considered key to students developing the competencies that autonomous work requires. To function autonomously in practice environments, it is argued that practitioners need to be able to judge the quality of their own work and the work of others. This is referred to as evaluative judgement. However, there is limited empirical literature relating to evaluative judgement and even less exploring the concept within occupational therapy. METHODS: This study used qualitative methods, seeking to understand the evaluative judgements of clinical practice made by third- and fourth-year occupational therapy students during practice education. RESULTS: Twenty-one interviews were conducted with third- (n = 10) and fourth-year occupational therapy students (n = 1), university support staff supporting practice education (n = 4), and practice education supervisors (n = 5) at one Australian university. Practice education grades and documentation were also used as data. Data were analysed thematically, and two themes, each with three sub-themes, were identified: students coming to understand expected standards, with the following sub-themes: students attuning to cues, cues that inform supervisors about students' meeting the standards, and barriers and frustrations to understanding standards; and practising and developing evaluative judgement, with the following sub-themes: making comparisons, acting on feedback, and reflective practice. CONCLUSIONS: Practice education experiences provide many context-specific opportunities for students to develop their evaluative judgement. Students may be supported to come to know what quality work looks like by offering scaffolded opportunities to develop evaluative judgement in university and practice education settings.

Student perspectives on factors that influence the implementation of evidence-based practice in occupational therapy.

AIM: Evidence-based practice (EBP) is perceived as an integral component of contemporary allied health practice. While allied health clinicians (such as occupational therapists) have generally positive attitudes towards EBP, research suggests that they find its implementation consistently challenging. The professional literature increasingly suggests that more effective EBP learning takes place when social constructivist approaches to learning are adopted. The authors of this study sought to use the pre-existing knowledge and perceptions of occupational therapy students to inform a curriculum review of an EBP unit. Therefore, the aim of this content analysis study was to investigate how occupational therapy students perceive and critically analyse factors that influence the implementation of EBP, specifically strengths, weaknesses, opportunities and threats (SWOT). METHODS: The study sampled students in the third year of a 4-year, Bachelor level Australian Occupational Therapy Program. The students were undertaking the first of two EBP units, and data were collected from the first assignment they submitted for these units. The assignment required the students to complete a SWOT analysis of EBP in occupational therapy, and response to a statement around whether it should be mandatory. Data were collected over 2 consecutive years from a total of 64 occupational therapy students. All data collected were subjected to content analysis, with themes identified by at least 25% of students in each cohort retained for analysis and formation into overall themes. RESULTS: Participating occupational therapy students identified five strengths, four weaknesses, six threats but only one opportunity for EBP. Three key themes were identified within the data: first, the role and purpose of EBP; second, the resources it requires and third, factors that influence its success. The students perceived the main purpose of EBP as supporting positive practice change, and highlighted its roles in building clinician capacity and professional credibility. They also characterized EBP as a separate and specialist practice that requires specific training and resourcing to achieve, and questioned its feasibility as an integrated part of daily practice. Along with previously identified challenges around time and resourcing, the students also highlighted negative attitudes and beliefs from senior colleagues as an important negative influence on the success of EBP. CONCLUSION: The current study consolidates a growing body of international literature about the value of social constructive approaches to EBP in undergraduate education. Vertical constructive alignment that embeds EBP through undergraduate education, rather than the provision of 'EBP'-specific units, should now be considered best practice. However, this embedded approach requires the explicit and repeated communication of the presence of EBP in all learning opportunities, to enable students to recognize opportunities to deploy their existing knowledge and skills.

Threshold concept acquisition in occupational therapy: A mixed methods study of students and clinicians.

BACKGROUND/AIM: Demand for occupational therapy graduates able to work in complex and diverse workplaces is increasing. The threshold concepts framework has emerged as one pathway to assist in the development of work-ready graduates. A previous Australian study identified 10 threshold concepts for occupational therapy; the aim of this study was to explore the acquisition of these. METHOD: A mixed-methods study using observational, cross sectional design and a triangulation design convergence model was undertaken. A survey using Likert scales and open response questions was developed by the authors. Seventy-three surveys were completed by 13 first and 20 final year Victorian occupational therapy students. Twenty recent graduates and 20 experienced clinicians from across Australia also participated. Seven first and 10 final year occupational therapy students, 10 recent graduates and 10 experienced clinicians also took part in in-depth interviews. Likert scales were used by participants to rate confidence that they had acquired the threshold concepts, they also answered open-response questions (via survey or interview) asking them to provide examples of how they would use threshold concepts in practice. RESULTS: Fourth year students were statistically significantly less confident than experienced clinicians for the threshold concepts Understanding the models and theories of occupational therapy and Applying clinical reasoning (P < 0.05). When whole group data were analysed, participants were most confident for Occupation and Client centred practice (P < 0.05). These results resonated with the qualitative findings which demonstrated a shift from surface to deep knowledge with increased experience. Fourth year students reported responses that suggested some threshold concepts were still in development, while experienced clinicians used a web of threshold concepts. CONCLUSION: This exploratory study provides provisional data to suggest that students appear to still be acquiring some threshold concepts at graduation and experienced clinicians make use of a web of threshold concepts in their clinical practice.

Economic evaluation of simulated and traditional clinical placements in occupational therapy education.

INTRODUCTION: This economic evaluation complements results of the randomised controlled trial that established non-inferiority of the learning outcomes of a one-week simulated clinical placement (SCP) in occupational therapy qualifying degrees in comparison to an equivalent traditional clinical placement (TCP). This companion study presents detailed cost analyses of two placement alternatives and a cost-benefit study to assess the value for money of SCP. An economic evaluation of simulated versus traditional placements has not previously been conducted in Australia. METHODS: Nine SCP/TCP rounds were conducted by six Australian universities. Costs were collected using study-specific instruments. Public health sector costs were sourced from available literature. Willingness-to-pay for SCP/TCP was estimated using both a Discrete Choice Experiment and a Contingent Valuation method. These methods were employed to assess a comparative 'value' of SCP/TCP from the perspective of heads of occupational therapy departments (N = 28), who were asked to put a monetary value on the broader range of benefits associated with SCP/TCP. RESULTS: From the universities' perspective the average cost per student ranged from AUD$460 to AUD$1511 for simulated and AUD$144 to AUD$1112 for traditional placement. From the health care sector perspective, the difference in costs favoured simulated placements for four implementations and traditional placements for five. In the Discrete Choice Experiment respondents preferred traditional rather than simulated placement and would pay additional AUD$533. The estimated monetary value of simulated placements from a contingent valuation ranged from AUD$200 to AUD$1600. CONCLUSIONS: For universities that procure TCPs predominately at public health care facilities and sustain high administrative overheads, the SCP program could be a cost-saving alternative. From a broader value-for-money perspective, respondents favoured TCP over SCP, yet placed importance on placement availability and opportunity to demonstrate competence for students during the placement. Results should be interpreted with caution and further research with larger sample sizes is required.

Simulated versus traditional occupational therapy placements: A randomised controlled trial.

BACKGROUND/AIM: Professional practise placements in occupational therapy education are critical to ensuring graduate competence. Australian occupational therapy accreditation standards allow up to 200 of a mandated 1000 placement hours to include simulation-based learning. There is, however, minimal evidence about the effectiveness of simulation-based placements compared to traditional placements in occupational therapy. We evaluated whether occupational therapy students completing a 40 hour (one week block) Simulated Clinical Placement (SCP) attained non-inferior learning outcomes to students attending a 40 hour Traditional Clinical Placement (TCP). METHODS: A pragmatic, non-inferiority, assessor-blinded, multicentre, randomised controlled trial involving students from six Australian universities was conducted. Statistical power analysis estimated a required sample of 425. Concealed random allocation was undertaken with a 1:1 ratio within each university. Students were assigned to SCP or TCP in one of three settings: vocational rehabilitation, mental health or physical rehabilitation. SCP materials were developed, manualised and staff training provided. TCPs were in equivalent practice areas. Outcomes were assessed using a standardised examination, unit grades, the Student Practice Evaluation Form-Revised and student confidence survey. A generalised estimating equation approach was used to assess non-inferiority of the SCP to the TCP. RESULTS: Of 570 randomised students (84% female), 275 attended the SCP and 265 the TCP (n = 540, 94.7% retention). There were no significant differences between the TCP and SCP on (i) examination results (marginal mean difference 1.85, 95% CI: 0.46-3.24; P = 0.087); (ii) unit score (mean (SD) SCP: 71.9 (8.8), TCP: 70.34 (9.1); P = 0.066); or (iii) placement fail rate, assessed using the Student Practice Evaluation Form-Revised (100% passed both groups). CONCLUSION: Students can achieve equivalent learning outcomes in a 40 hour simulated placement to those achieved in a 40 hour traditional placement. These findings provide assurance to students, educators and professional accreditation bodies that simulation can be embedded in occupational therapy education with good effect.

Effectiveness and cost-effectiveness of embedded simulation in occupational therapy clinical practice education: study protocol for a randomised controlled trial.

BACKGROUND: Clinical placements are a critical component of the training for health professionals such as occupational therapists. However, with growing student enrolments in professional education courses and workload pressures on practitioners, it is increasingly difficult to find sufficient, suitable placements that satisfy program accreditation requirements. The professional accrediting body for occupational therapy in Australia allows up to 200 of the mandatory 1000 clinical placement hours to be completed via simulation activities, but evidence of effectiveness and efficiency for student learning outcomes is lacking. Increasingly placement providers charge a fee to host students, leading educators to consider whether providing an internal program might be a feasible alternative for a portion of placement hours. Economic analysis of the incremental costs and benefits of providing a traditional versus simulated placement is required to inform decision-making. METHODS/DESIGN: This study is a pragmatic, non-inferiority, single-blind, multicentre, two-group randomised controlled trial (RCT) with an embedded economic analysis. The RCT will compare a block of 40 hours of simulated placement (intervention) with a 40-hour block of traditional placement (comparator), with a focus on student learning outcomes and delivery costs. Six universities will instigate the educational intervention within their respective occupational therapy courses, randomly assigning their cohort of students (1:1 allocation) to the simulated or traditional clinical placements. The primary outcome is achievement of professional behaviours (e.g. communication, clinical reasoning) as assessed by a post-placement written examination. Secondary outcomes include proportions passing the placement assessed using the Student Practice Evaluation Form-Revised, changes in student confidence pre-/post-placement, student and educator evaluation of the placement experience and cost-effectiveness of simulated versus traditional clinical placements. Comprehensive cost data will be collected for both the simulated and traditional placement programs at each site for economic evaluation. DISCUSSION: Use of simulation in health-related fields like occupational therapy is common, but these activities usually relate to brief opportunities for isolated skill development. The simulated clinical placement evaluated in this trial is less common because it encapsulates a 5-day block of integrated activities, designed and delivered in a manner intended to emulate best-practice placement experiences. The planned study is rare due to inclusion of an economic analysis that aims to provide valuable information about the relationship between costs and outcomes across participating sites. TRIAL REGISTRATION: Australian New Zealand Clinical Trials Registry, ACTRN12616001339448 . Registered 26 September 2016.

Instructional practices for evidence-based practice with pre-registration allied health students: a review of recent research and developments.

The aim of this study is to update a previous review published in this journal on the effectiveness of teaching and assessment interventions for evidence based practice in health professions, and to determine the extent to which the five recommendations made from that review have been implemented. The Integrating Theory, Evidence and Action method was used to synthesise all published evidence from 2011 to 2015, which addressed instructional practices used for evidence based practice with pre-registration allied health students. Seventeen articles were found to meet the inclusion criteria, and were analysed for both their individual rigour and relationship to the five recommendations. The evidence reviewed in this study was diverse in both its geographical setting and the allied health disciplines represented. Most of the evidence used less rigorous methods, and the evidence base is generally exploratory in nature. To date, the five recommendations regarding instructional practices in this area have been implemented to varying degrees. Many current practices promote social negotiation, collaborative decision-making and collaborative learning, so the social constructivist approach is being adopted. However, the prior knowledge of students is not being assessed as a basis for scaffolding, communication of evidence based practice to varying audiences is rarely addressed and the role of clinicians in the learning of evidence based practice knowledge, skills, beliefs and attitudes remains limited.

Transformation from student to occupational therapist: Using the Delphi technique to identify the threshold concepts of occupational therapy.

BACKGROUND/AIM: Understanding and facilitating the transformation from occupational therapy student to practitioner is central to the development of competent and work-ready graduates. However, the pivotal concepts and capabilities that need to be taught and learnt in occupational therapy are not necessarily explicit. The threshold concepts theory of teaching and learning proposes that every discipline has a set of transformational concepts that students must acquire in order to progress. As students acquire the threshold concepts, they develop a transformed way of understanding content related to their course of study which contributes to their developing expertise. The aim of this study was to identify the threshold concepts of occupational therapy. METHOD: The Delphi technique, a data collection method that aims to demonstrate consensus in relation to important questions, was used with three groups comprising final year occupational therapy students (n = 11), occupational therapy clinicians (n = 21) and academics teaching occupational therapy (n = 10) in Victoria, Australia. RESULTS: Participants reached consensus regarding 10 threshold concepts for the occupational therapy discipline. These are: understanding and applying the models and theories of occupational therapy; occupation; evidence-based practice; clinical reasoning; discipline specific skills and knowledge; practising in context; a client-centred approach; the occupational therapist role; reflective practice and; a holistic approach. CONCLUSION: The threshold concepts identified provide valuable information for the discipline. They can potentially inform the development of competencies for occupational therapy and provide guidance for teaching and learning activities to facilitate the transformation to competent practitioner.