Establishing the Predictive Validity of the Assessment of Motor and Process Skills for Driving Performance Outcomes.

IMPORTANCE: Although the Assessment of Motor and Process Skills (AMPS) is an excellent tool for evaluating the functional performance of instrumental activities of daily living (IADLs), a limited number of studies have used the AMPS for decisions regarding the IADL of fitness to drive and community mobility. OBJECTIVE: To determine the specificity and sensitivity of the AMPS as a tool for determining a person's fitness to drive. DESIGN: Cross-sectional observational design. SETTING: Three driving rehabilitation programs in three states. PARTICIPANTS: Participants were 388 community-living adults (M age = 68.74 yr, SD = 11.53); 196 adults were recruited before completing a comprehensive driving evaluation, and 192 were recruited in two other studies of older drivers. OUTCOME AND MEASURES: AMPS and results of comprehensive driving evaluation or on-road assessment. RESULTS: Using a logistical regression, AMPS Motor and Process Skills scores yielded a sensitivity of 84.6% and a specificity of 88.8%. The odds ratio of the AMPS Motor Skills score was .347; for the AMPS Process Skills score, it was .014. Using cross-validations, the model with AMPS Motor and Process scores produced a cross-validation area under the curve of .918, with sensitivity and specificity of 84.6% and 88.4%, respectively, and a probability greater than .334 was used for predicting a fail or drive-with-restriction evaluation. CONCLUSIONS AND RELEVANCE: The AMPS Motor and Process Skills scores revealed significant differences between those who failed or had driving restrictions and with those who passed the driving evaluation, which supported the AMPS as an effective tool for predicting fitness to drive. Plain-Language Summary: This study demonstrates how the Assessment of Motor and Process Skills (AMPS), as a top-down occupational therapy assessment tool, can be used to differentiate between medically at-risk drivers who are likely to pass a comprehensive driving evaluation and those who are likely to fail or need restrictions. AMPS will assist occupational therapy practitioners in determining who is most appropriate to receive driving rehabilitation services and/or when to refer a person for a comprehensive driving evaluation.

Improving Functional Outcomes for Vascular Amputees Through Use of Mirror Therapy and Elimination of the Effects of Electromagnetic Fields.

The objective of this pilot study was to investigate the effectiveness of combining an amputee limb cover to eliminate the effects of electromagnetic fields (i.e., pain) and a Mirror Therapy exercise program to improve functional outcomes for vascular amputees. A cross-sectional repeated-measures design was used with 14 participants with either acute amputations or surgery at least 8 to 24 months previously. The 4-week intervention included the use of an amputee limb cover and mirror therapy exercises each day. The outcome measures were activities of daily living interference (e.g., self-care, walking, car transfer, low chair transfer, sleep), and well-being (e.g., satisfaction, mood, quality of life) at three times (pre- and posttreatment and maintenance). Participants with acute amputations made significant improvements in the areas of self-care, walking, car transfer, sleep, mood, and quality of life, while the subacute participants improved significantly in sleep and satisfaction. A reduction in the time required before prosthetic fitting decreased from 12 weeks to 8 weeks for acute amputees and an improvement in wearing tolerance from 0-2 to 8-12 hours for the subacute amputees were unexpected results suggesting the combined intervention may improves the extent to which amputees can increase participation in their activities of everyday living.

Participants' preference for type of leaflet used to feed back the results of a randomised trial: a survey.

BACKGROUND: Hundreds of thousands of volunteers take part in medical research, but many will never hear from researchers about what the study revealed. There is a growing demand for the results of randomised trials to be fed back to research participants both for ethical research practice and for ensuring their co-operation in a trial. This study aims to determine participants' preferences for type of leaflet (short versus long) used to summarise the findings of a randomised trial; and to test whether certain characteristics explained participants' preferences. METHODS: 553 participants in a randomised trial about General Practitioners' access to Magnetic Resonance Imaging for patients presenting with suspected internal derangement of the knee were asked in the final follow-up questionnaire whether they would like to be fed back the results of the trial. Participants who agreed to this were included in a postal questionnaire survey asking about their preference, if any, between a short and a long leaflet and what it was about the leaflet that they preferred. Multinomial logistic regression was used to test whether certain demographics of responding participants along with treatment group explained whether a participant had a preference for type of leaflet or no preference. RESULTS: Of the participants who returned the final follow-up questionnaire, 416 (88%) agreed to receive the results of the trial. Subsequently 132 (32%) participants responded to the survey. Most participants preferred the longer leaflet (55%) and the main reasons for this were the use of technical information (94%) and diagrams (89%). There was weak evidence to suggest that gender might explain whether participants have a preference for type of leaflet or not (P = 0.084). CONCLUSIONS: Trial participants want to receive feed back about the results and appear to prefer a longer leaflet. Males and females might require information to be communicated to them differently and should be the focus of further research. TRIAL REGISTRATION: The trial is registered with http://www.isrctn.org/ and ID is ISRCTN76616358.

Some effects of teaching undergraduate medical students on general practitioner thinking and learning.

BACKGROUND: This study examined the processes of thinking and learning undertaken by general practitioners (GPs, family physicians) brought about by the presence of undergraduate medical students. METHOD: A qualitative study was undertaken using critical incident templates (a novel approach in this context) and focus groups. Data were analysed thematically, with iterative discussion between members of the study team after independent analysis. RESULTS: Thirty-eight critical incident templates were returned (40%), of which 27 (28%) were completed. Three themes were identified: challenges to the GP's thinking or learning behaviour (these included having to justify actions and answer questions); changes to the dynamic between doctor and patient (listening to discussion of their condition between doctor and student was reported to change the patient's experience); and additions to the GP's workload. Teaching made GPs' day longer and sometimes caused friction with colleagues. CONCLUSIONS: Our data elucidate a range of thinking and learning activities that are brought about by the presence of a student in a GP's consulting room. They show that there are a number of challenges that stimulate these activities. We recommend that undergraduate teaching in general practice be given greater recognition for the stimulus it provides for continuing professional development.

A question of balance: a qualitative study of mothers' interpretations of dietary recommendations.

PURPOSE: Poor nutrition is an increasing problem for economically deprived families, and mothers play a key role in establishing children's diets. We explored mothers' understanding of health-promotion recommendations for healthy eating. METHODS: We conducted qualitative semistructured interviews of 46 mothers within a relatively socioeconomically deprived community. Data were subject to framework analysis. RESULTS: The basic slogans of health promotion were known by mothers and had been adopted into everyday language. Television was the main source of information on dietary advice. Barriers to making changes were due to practical constraints (time, money, family preferences) and a desire to enhance quality of life through enjoyment of food. Although the headline messages of the importance of a healthy diet and what constitutes a healthy diet had reached mothers, misunderstandings were common and led to inappropriate actions. Many descriptions of what mothers reported as a balanced diet would not satisfy official definitions. Some women willfully adapted their understanding of advice to suit their preferences and the reality of their lives and family circumstances. CONCLUSIONS: To provide effective advice and guidance, health professionals need a deeper understanding of how families interpret messages about healthy eating. In addition, they should provide advice based on more individual understandings of diet and take into account patient preferences and life circumstances.

An evaluation of the sources and availability of careers advice in UK medical schools.

This article reports a study evaluating the sources and availability of career advice available to students in UK medical schools in order to ascertain the need for an enhanced role for careers information in undergraduate curricula.

Evaluation of different delivery modes of an interactive e-learning programme for teaching cultural diversity.

OBJECTIVE: UK medical schools find it challenging to provide standardised teaching to expanding year intakes. In addition, developing and implementing diversity training can cause difficulties. This paper describes the evaluation of an interactive e-learning programme to raise awareness and understanding of communication difficulties in diversity consultations. METHODS: The programme was part of an undergraduate portfolio-based community module. Three hundred and two students were assigned to one of three delivery methods--a large group setting, small groups with a facilitator, and as part of distance learning while on community placement. The evaluation included analysis of their coursework marks, a self-completed evaluation questionnaire, and small group discussions. RESULTS: Two hundred and twenty-three students took part in the evaluation. They were able to apply the concepts they learnt to clinical examples from their own experiences. Type of delivery did not affect coursework marks, but students tended to prefer the e-learning as part of a distance learning package. They offered helpful suggestions to improve its complexity and range. CONCLUSION: The acceptability and utility of this e-learning module both in face to face teaching and remote placement has been demonstrated, and evaluation by the students has provided valuable information for its further development. PRACTICE IMPLICATIONS: All medical schools should include some diversity training, and further research should concentrate on the effects of this type of learning on longer term outcomes such as attitude and performance tests. Such tools could reduce demands on staff time in facilitation of small groupwork, and their cost effectiveness could be increased by making them available to other medical schools.

Using postal randomization to replace telephone randomization had no significant effect on recruitment of patients.

OBJECTIVE: To test the effect of postal randomization on recruitment of patients into a randomized trial in primary care. STUDY DESIGN AND SETTING: General practices used a telephone service to randomize patients in our trial. Delays in the start of recruitment at some sites led us to modify the randomization procedure. When new practices took part patients completed and posted baseline materials to the Trial Secretary in York who performed the randomization and informed those concerned of the allocation. RESULTS: Of the 647 practices who were invited to take part, 130 (45%) of 288 agreed to participate using telephone randomization and 155 (43%) of 359 using the postal method. These practices recruited 553 patients from November 2002 to October 2004 across 11 sites in the United Kingdom. The postal method reduced the number of patients recruited by a factor of 0.86 (95% confidence interval=0.62-1.20), or 14%. The number of general practitioners working in a practice significantly increased patient recruitment by a factor of 1.12 (1.05-1.20), whereas practice distance from hospital significantly decreased recruitment by a factor of 0.98 (0.97-0.99). CONCLUSION: Postal randomization had no significant effect on recruitment of patients into our trial.

Exploring students' perceptions on the use of significant event analysis, as part of a portfolio assessment process in general practice, as a tool for learning how to use reflection in learning.

BACKGROUND: Portfolio learning enables students to collect evidence of their learning. Component tasks making up a portfolio can be devised that relate directly to intended learning outcomes. Reflective tasks can stimulate students to recognise their own learning needs. Assessment of portfolios using a rating scale relating to intended learning outcomes offers high content validity. This study evaluated a reflective portfolio used during a final-year attachment in general practice (family medicine). Students were asked to evaluate the portfolio (which used significant event analysis as a basis for reflection) as a learning tool. The validity and reliability of the portfolio as an assessment tool were also measured. METHODS: 81 final-year medical students completed reflective significant event analyses as part of a portfolio created during a three-week attachment (clerkship) in general practice (family medicine). As well as two reflective significant event analyses each portfolio contained an audit and a health needs assessment. Portfolios were marked three times; by the student's GP teacher, the course organiser and by another teacher in the university department of general practice. Inter-rater reliability between pairs of markers was calculated. A questionnaire enabled the students' experience of portfolio learning to be determined. RESULTS: Benefits to learning from reflective learning were limited. Students said that they thought more about the patients they wrote up in significant event analyses but information as to the nature and effect of this was not forthcoming. Moderate inter-rater reliability (Spearman's Rho .65) was found between pairs of departmental raters dealing with larger numbers (20-60) of portfolios. Inter-rater reliability of marking involving GP tutors who only marked 1-3 portfolios was very low. Students rated highly their mentoring relationship with their GP teacher but found the portfolio tasks time-consuming. CONCLUSION: The inter-rater reliability observed in this study should be viewed alongside the high validity afforded by the authenticity of the learning tasks (compared with a sample of a student's learning taken by an exam question). Validity is enhanced by the rating scale which directly connects the grade given with intended learning outcomes. The moderate inter-rater reliability may be increased if a portfolio is completed over a longer period of time and contains more component pieces of work. The questionnaire used in this study only accessed limited information about the effect of reflection on students' learning. Qualitative methods of evaluation would determine the students experience in greater depth. It would be useful to evaluate the effects of reflective learning after students have had more time to get used to this unfamiliar method of learning and to overcome any problems in understanding the task.

Improving response rates using a monetary incentive for patient completion of questionnaires: an observational study.

BACKGROUND: Poor response rates to postal questionnaires can introduce bias and reduce the statistical power of a study. To improve response rates in our trial in primary care we tested the effect of introducing an unconditional direct payment of 5 pounds for the completion of postal questionnaires. METHODS: We recruited patients in general practice with knee problems from sites across the United Kingdom. An evidence-based strategy was used to follow-up patients at twelve months with postal questionnaires. This included an unconditional direct payment of 5 pounds to patients for the completion and return of questionnaires. The first 105 patients did not receive the 5 pounds incentive, but the subsequent 442 patients did. We used logistic regression to analyse the effect of introducing a monetary incentive to increase the response to postal questionnaires. RESULTS: The response rate following reminders for the historical controls was 78.1% (82 of 105) compared with 88.0% (389 of 442) for those patients who received the 5 pounds payment (diff = 9.9%, 95% CI 2.3% to 19.1%). Direct payments significantly increased the odds of response (adjusted odds ratio = 2.2, 95% CI 1.2 to 4.0, P = 0.009) with only 12 of 442 patients declining the payment. The incentive did not save costs to the trial--the extra cost per additional respondent was almost 50 pounds. CONCLUSION: The direct payment of 5 pounds significantly increased the completion of postal questionnaires at negligible increase in cost for an adequately powered study.

The DAMASK trial protocol: a pragmatic randomised trial to evaluate whether GPs should have direct access to MRI for patients with suspected internal derangement of the knee.

BACKGROUND: Though new technologies like Magnetic Resonance Imaging (MRI) may be accurate, they often diffuse into practice before thorough assessment of their value in diagnosis and management, and of their effects on patient outcome and costs. MRI of the knee is a common investigation despite concern that it is not always appropriate. There is wide variation in general practitioners (GPs) access to, and use of MRI, and in the associated costs. The objective of this study was to resolve uncertainty whether GPs should refer patients with suspected internal derangement of the knee for MRI or to an orthopaedic specialist in secondary care. METHODS/DESIGN: The design consisted of a pragmatic multi-centre randomised trial with two parallel groups and concomitant economic evaluation. Patients presenting in general practice with suspected internal derangement of the knee and for whom their GP was considering referral to an orthopaedic specialist in secondary care were eligible for inclusion. Within practices, GPs or practice nurses randomised eligible and consenting participants to the local radiology department for an MRI examination, or for consultation with an orthopaedic specialist. To ensure that the waiting time from GP consultation to orthopaedic appointment was similar for both trial arms, GPs made a provisional referral to orthopaedics when requesting the MRI examination. Thus we evaluated the more appropriate sequence of events independent of variations in waiting times. Follow up of participants was by postal questionnaires at six, twelve and 24 months after randomisation. This was to ensure that the evaluation covered all events up to and including arthroscopy. DISCUSSION: The DAMASK trial should make a major contribution to the development of evidence-based partnerships between primary and secondary care professionals and inform the debate when MRI should enter the diagnostic pathway.

Students' views of reflective learning techniques: an efficacy study at a UK medical school.

OBJECTIVE: To describe the effects of a voluntary intervention using reflective learning techniques on students' learning. DESIGN: An interventional study with reflective learning techniques offered to medical students. SETTING: Year 3 of undergraduate medicine at Cardiff University where the curriculum is integrated with early clinical contact. PARTICIPANTS: All 232 Year 3 students were invited to participate. A total of 65 attended an introductory lecture. After the lecture 35 students agreed to take part; 15 of these subsequently dropped out (some before attending tutorial groups, others after taking part for some weeks). INTERVENTIONS: Participants kept learning journals for 2 terms and attended fortnightly, facilitated tutorial groups where they discussed their reflective journal entries. Main outcome measures were qualitative interviews and examination results. RESULTS: Interviews were carried out with 19 full participants, 4 initial participants and 7 non-participants. Participants perceived that they gained a greater ability to identify learning objectives and to integrate learning. The tutorial groups encouraged students to compare progress with their peers. Some students did not take part because they thought that the large factual content of the curriculum would make reflective learning less useful. There were no differences between the groups in examination results. CONCLUSIONS: Students among the small, self-selected group of participants were better able to identify what they needed to learn although there was no improvement in examination results. Students appear unlikely to take up voluntary reflective learning if they do not think it relates to the curriculum and assessments. Student culture exerts a potent effect on willingness to attend extra tutorial groups.

Learning to mark: a qualitative study of the experiences and concerns of medical markers.

BACKGROUND: Although there is published research on the methods markers use in marking various types of assessment, there is relatively little information on the processes markers use in approaching a marking exercise. This qualitative paper describes the preparation and experiences of general practice (GP) teachers who undertake marking a written assessment in an undergraduate medical course. METHODS: Semi-structured interviews were conducted with seven of the 16 GP tutors on an undergraduate course. The purposive sample comprised two new markers, two who had marked for a couple of years and three experienced markers. Each respondent was interviewed twice, once following a formative assessment of a written case study, and again after a summative assessment. All interviews were audio-taped and analysed for emerging themes. A respondent validation exercise was conducted with all 16 GP tutors. RESULTS: Markers had internal concerns about their ability to mark fairly and made considerable efforts to calibrate their marking. They needed guidance and coaching when marking for the first time and adopted a variety of marking styles, reaching a decision through a number of routes. Dealing with pass/fail borderline scripts and the consequences of the mark on the student were particular concerns. Even experienced markers felt the need to calibrate their marks both internally and externally CONCLUSION: Previous experience of marking appears to improve markers' confidence and is a factor in determining the role which markers adopt. Confidence can be improved by giving clear instructions, along with examples of marking. The authors propose that one method of providing this support and coaching could be by a process of peer review of a selection of papers prior to the main marking. New markers in particular would benefit from further guidance, however they are influenced by others early on in their marking career and course organisers should be mindful of this when arranging double marking.

Previously unidentified morbidity in patients with intellectual disability.

BACKGROUND: Adults with a learning disability frequently have unmet health needs. The cause for this is complex and may be related to difficulties in accessing usual primary care services. Health checks have been widely recommended as a solution to this need. AIM: To determine the likelihood that a structured health check by the primary care team supported by appropriate education would identify and treat previously unrecognised morbidity in adults with an intellectual disability. DESIGN OF STUDY: Individuals were identified within primary care teams and a structured health check performed by the primary care team. This process was supported by an educational resource. Face-to-face audit with the team was performed 3 months following the check. SETTING: Forty general practices within three health authorities in south and mid-Wales participated. They had a combined registered patient population of 354 000. METHOD: Health checks were conducted for 190 (60%) of 318 identified individuals; 128 people moved, died, withdrew from the study, or refused to participate. RESULTS: Complete data were available on 181 health checks; 51% had new needs recognised, of whom 63% had one health need, 25% two health needs, and 12% more than two. Sixteen patients (9%) had serious new morbidity discovered. Management had been initiated for 93% of the identified health needs by the time of audit. This study is the first to identify new disease findings in a primary care population and the likelihood that such disease will be treated. CONCLUSIONS: The findings reflect a concern that current care delivery leaves adults with an intellectual disability at risk of both severe and milder illness going unrecognised. Health checks present one mechanism for identifying and treating such illness in primary care.

Communication, stress and distress: evolution of an individual support programme for medical students and doctors.

BACKGROUND: The stress and distress that can be associated with poor communication is well recognised. The problem of how to manage this effectively has yet to be solved. METHODS: In December 2001, the Communication Skills Unit sited in the Department of General Practice in the University of Wales, College of Medicine set up a pilot project for medical students and qualified doctors who were perceived to have communication difficulties. Referrals were made via the deaneries following an agreed format. Initial assessments were used to build an individually tailored programme addressing clients' needs. DISCUSSION: Throughout the year a variety of problems were encountered ranging from language difficulties to bullying and mental health issues. Anecdotal evidence of improvements has been clear, but measuring the outcome of this intervention has proved to be more challenging. Our efforts are now focused on refining assessment and evaluation. This type of service can provide the link between national and local initiatives as well as helping to develop strategies for assessment and training in communication.

Putting double marking to the test: a framework to assess if it is worth the trouble.

BACKGROUND: It is a challenge to assign a mark that accurately measures the quality of students' work in essay-type assessments that require an element of judgement and fairness by the markers. Double marking such assessments has been seen as a way of improving the reliability of the mark. The analysis approach often taken is to look for absolute agreement between markers instead of looking at all aspects of reliability. AIM: To develop an analytic process that will examine the components and meanings of reliability calculations that can be used for assessing the value of double marking a piece of work. METHODS: An undergraduate case study assessment in General Practice was used as an illustration. Datasets of double marking were collected retrospectively for 1999-2000, and prospectively for 2002-03. An assessment of intermarker agreement and its effect on the reliability of the final mark for students was made, using methods dependent on the type of data collected and Generalisability Theory. RESULTS AND CONCLUSIONS: The data were used to illustrate how to interpret the results of Bland and Altman plots, anova tables and Cohen's kappa calculations. Generalisability Theory was used to show that, while there was reasonable agreement between markers, the reliability of the mark for the student was still only moderate, probably due to unexplained variability elsewhere in the process. Possible reasons for this variability are discussed. A flowchart of the decisions and actions needed to judge whether a piece of work should be double marked has been constructed.

Shared decision making and risk communication in practice: a qualitative study of GPs' experiences.

BACKGROUND: Important barriers to the wider implementation of shared decision making remain. The experiences of professionals who are skilled in this approach may identify how to overcome these barriers. AIMS: To identify the experiences and views of professionals skilled in shared decision making and risk communication, exploring the opportunities and challenges for implementation. DESIGN OF STUDY: Qualitative study. SETTING: Gwent Health Authority. METHOD: Exit interviews using focus group methodology with 20 GPs who had been in practice between 1 and 10 years, and participated in an explanatory trial lasting 6 months. The trial interventions comprised training in shared decision-making skills and the use of risk communication materials. The doctors consulted with up to 48 patients each (mean = 40, half of them audiotaped) for the study. RESULTS: The GPs indicated positive attitudes towards involving patients and described positive effects on their consultations. However, the frequency of applying the new skills and tools was limited outside the trial. Doctors were selective about when they felt greater patient involvement was appropriate and feasible, rather than seeking to apply the approaches to the majority of consultations. They felt they often responded to consumer preferences for low levels of involvement in decision making. Time limitations were important in not implementing the approach more widely. CONCLUSION: The promotion of 'patient involvement' appears likely to continue. Professionals appear receptive to this, and willing to acquire the relevant skills. Strategies for wider implementation of patient involvement could address how consultations are scheduled in primary care, and raise consumers' expectations or desires for involvement.

Predicting the clinical course of suspected acute viral upper respiratory tract infection in children.

BACKGROUND: Suspected acute viral upper respiratory tract infection (SAVURTI) is the commonest acute reason why children consult in general practice. The clinical course varies widely and about one in five children re-consult for the same SAVURTI episode. If clinicians had feasible tools for predicting which children are likely to suffer a prolonged course, then additional explanations and possibly treatments could be provided at the initial consultation that might enable carers to manage the condition without re-consulting. OBJECTIVE: To identify features available on the day of consulting that might predict a prolonged clinical course among children with SAVURTI. METHOD: Regression analysis using Canadian Respiratory Illness and Flu Scale (CARIFS) data from a randomized controlled trial cohort of children aged from 6 months to 12 years consulting in general practice with SAVURTI. RESULTS: Two variables from the clinician's records ('age' and 'cough') and two variables from the CARIFS completed by carers on the day of consulting ('fever' and 'low energy, tired') explained approximately 15% of the variation present in CARIFS scores on day seven. CONCLUSION: Children and carers may benefit from a clear account of the evidence that the clinical course of RTIs in children varies widely and may be longer that expected, and that prediction for individuals is difficult.

Assessment of professional attitude and conduct in medical undergraduates.

One year after implementation of a new schedule of summative assessment of professional attitude and conduct (AC) for medical undergraduates at the University of Wales College of Medicine (UWCM), we evaluated its effectiveness and obtained feedback of the perceptions of participating teachers and students. An anonymous questionnaire was administered to all 4th year students and their clinical teachers. Four out of 180 students actually failed to complete the 4th year as a result of unsatisfactory AC. Of these, three students were identified as having a problem which benefited from supportive remediation. One hundred and twenty out of 244 (49%) teachers and 166 out of 195 (85%) students completed the feedback questionnaires. The majority of students and teachers agreed that AC should be an integral part of their assessment. However, several problems were highlighted, in particular, students' perception that marks were sometimes based on assessment of knowledge rather than attitude and conduct and teachers' reluctance to identify unprofessional behaviour in students. The use of longitudinal assessment of professional AC was found to be effective at identifying students with persistent AC problems and also provided an opportunity for formative experience. The reluctance of teachers to identify negative behaviours, compared to deficiencies in skill and knowledge, is an important aspect of developing a reliable method of evaluating professional attitudes.