Functional Interviewing Was Associated With Improved Agreement Among Expert Psychiatrists in Estimating Claimant Work Capacity: A Secondary Data Analysis of Real-Life Work Disability Evaluations.

OBJECTIVE: Functional evaluations establish functional and work (in-)capacities in the context of disability assessments and are increasingly recommended as a modern technique for work disability assessments. The RELY (Reliable disability EvaLuation in psychiatrY)-studies introduced semi-structured functional interviews in real-life assessments of claimants with mental disorders for evaluating their self-perceived health-related limitations and for investigating the reproducibility of work capacity (WC) estimates. Functional interviews elicit claimants' self-perceptions about their work-related limitations and capacities in the labour market. This secondary data analysis explored the coverage of work-related key topics in these interviews and investigated whether interviews with high coverage (versus low coverage) of work-related topics resulted in better reproducibility of WC estimates among experts. METHODS: Thirty video-taped RELY-assessments underwent a content analysis along a predefined framework for functional interviewing, including the claimant's self-perceived work limitations and work-related health complaints as centrepieces of functional interviewing. Following transcription, interviews were segmented into coding units. Coding units were allocated to the five steps with 19 key topics of the framework. Enquiry into key topics was ascertained by summing the functional coding units per key topic. Median split grouped the interviews into high and low coverage of functional topics and compared them for inter-rater reliability (intraclass correlation coefficient, ICC) and inter-rater agreement (standard error of measurement, SEM). RESULTS: Interviews were broken down in 40,010 coding units, 31% of which addressed functional topics. Enquiries in self-perceived work limitations and work-related health complaints were sparse (coding units medianpsychiatrist between 0 and 1.5, medianpatients between 0 and 9.5). High coverage interviews enquired on more functional topics (68% vs. 42%, chi2(1, N = 38) = 5.32, p = 0.021) and in more depth (36% vs. 16% of functional coding units, chi2(1, N = 1,314) = 141.15, p < 0.001). Interviews with higher functional coverage reached significantly higher inter-rater agreement in WC ratings among experts (mean difference in SEM, low-high coverage, 7.5% WC, 95% CI 0.2 to 15.1%WC). Inter-rater reliability was low in both groups (ICC, 0.38 versus 0.40). CONCLUSIONS: Content analysis showed little enquiry by experts on claimants' self-perceived activity limitations and work-related capacity. The association between interviews with higher functional coverage and better expert agreement on the claimants' remaining WC requires confirmation in prospective studies.

The reproducibility of psychiatric evaluations of work disability: two reliability and agreement studies.

BACKGROUND: Expert psychiatrists conducting work disability evaluations often disagree on work capacity (WC) when assessing the same patient. More structured and standardised evaluations focusing on function could improve agreement. The RELY studies aimed to establish the inter-rater reproducibility (reliability and agreement) of 'functional evaluations' in patients with mental disorders applying for disability benefits and to compare the effect of limited versus intensive expert training on reproducibility. METHODS: We performed two multi-centre reproducibility studies on standardised functional WC evaluation (RELY 1 and 2). Trained psychiatrists interviewed 30 and 40 patients respectively and determined WC using the Instrument for Functional Assessment in Psychiatry (IFAP). Three psychiatrists per patient estimated WC from videotaped evaluations. We analysed reliability (intraclass correlation coefficients [ICC]) and agreement ('standard error of measurement' [SEM] and proportions of comparisons within prespecified limits) between expert evaluations of WC. Our primary outcome was WC in alternative work (WCalternative.work), 100-0%. Secondary outcomes were WC in last job (WClast.job), 100-0%; patients' perceived fairness of the evaluation, 10-0, higher is better; usefulness to psychiatrists. RESULTS: Inter-rater reliability for WCalternative.work was fair in RELY 1 (ICC 0.43; 95%CI 0.22-0.60) and RELY 2 (ICC 0.44; 0.25-0.59). Agreement was low in both studies, the 'standard error of measurement' for WCalternative.work was 24.6 percentage points (20.9-28.4) and 19.4 (16.9-22.0) respectively. Using a 'maximum acceptable difference' of 25 percentage points WCalternative.work between two experts, 61.6% of comparisons in RELY 1, and 73.6% of comparisons in RELY 2 fell within these limits. Post-hoc secondary analysis for RELY 2 versus RELY 1 showed a significant change in SEMalternative.work (- 5.2 percentage points WCalternative.work [95%CI - 9.7 to - 0.6]), and in the proportions on the differences ≤ 25 percentage points WCalternative.work between two experts (p = 0.008). Patients perceived the functional evaluation as fair (RELY 1: mean 8.0; RELY 2: 9.4), psychiatrists as useful. CONCLUSIONS: Evidence from non-randomised studies suggests that intensive training in functional evaluation may increase agreement on WC between experts, but fell short to reach stakeholders' expectations. It did not alter reliability. Isolated efforts in training psychiatrists may not suffice to reach the expected level of agreement. A societal discussion about achievable goals and readiness to consider procedural changes in WC evaluations may deserve considerations.

The assessment of work endurance in disability evaluations across European countries.

PURPOSE: Chronic disease is often associated with a reduced energy level, which limits the capacity to work full-time. This study aims to investigate whether the construct work endurance is part of disability assessment in European countries and what assessment procedures are used. We defined work endurance as the ability to sustain working activities for a number of hours per day and per week. MATERIALS AND METHODS: We conducted a survey using two self-constructed questionnaires. We addressed 35 experts from 19 countries through the European Union of Medicine in Assurance and Social Security (EUMASS). We gathered descriptive data on various aspects of (the assessment of) work endurance. RESULTS: Experts from 16 countries responded. In most countries work endurance is assessed. We found few professional guidelines specific for the assessment of work endurance. Both somatic and mental diseases may cause limited work endurance. Methods to assess work endurance vary, objective methods rating as most suitable. Almost half of the countries report controversies on the assessment of work endurance. CONCLUSIONS: Work endurance is recognised and assessed as an aspect of work disability assessment in Europe. However, controversies exist and evidence based guidelines, including reliable and valid methods to assess work endurance, are lacking.

Inter-rater agreement in evaluation of disability: systematic review of reproducibility studies.

OBJECTIVES:  To explore agreement among healthcare professionals assessing eligibility for work disability benefits. DESIGN:  Systematic review and narrative synthesis of reproducibility studies. DATA SOURCES:  Medline, Embase, and PsycINFO searched up to 16 March 2016, without language restrictions, and review of bibliographies of included studies. ELIGIBILITY CRITERIA:  Observational studies investigating reproducibility among healthcare professionals performing disability evaluations using a global rating of working capacity and reporting inter-rater reliability by a statistical measure or descriptively. Studies could be conducted in insurance settings, where decisions on ability to work include normative judgments based on legal considerations, or in research settings, where decisions on ability to work disregard normative considerations. : Teams of paired reviewers identified eligible studies, appraised their methodological quality and generalisability, and abstracted results with pretested forms. As heterogeneity of research designs and findings impeded a quantitative analysis, a descriptive synthesis stratified by setting (insurance or research) was performed. RESULTS:  From 4562 references, 101 full text articles were reviewed. Of these, 16 studies conducted in an insurance setting and seven in a research setting, performed in 12 countries, met the inclusion criteria. Studies in the insurance setting were conducted with medical experts assessing claimants who were actual disability claimants or played by actors, hypothetical cases, or short written scenarios. Conditions were mental (n=6, 38%), musculoskeletal (n=4, 25%), or mixed (n=6, 38%). Applicability of findings from studies conducted in an insurance setting to real life evaluations ranged from generalisable (n=7, 44%) and probably generalisable (n=3, 19%) to probably not generalisable (n=6, 37%). Median inter-rater reliability among experts was 0.45 (range intraclass correlation coefficient 0.86 to κ-0.10). Inter-rater reliability was poor in six studies (37%) and excellent in only two (13%). This contrasts with studies conducted in the research setting, where the median inter-rater reliability was 0.76 (range 0.91-0.53), and 71% (5/7) studies achieved excellent inter-rater reliability. Reliability between assessing professionals was higher when the evaluation was guided by a standardised instrument (23 studies, P=0.006). No such association was detected for subjective or chronic health conditions or the studies' generalisability to real world evaluation of disability (P=0.46, 0.45, and 0.65, respectively). CONCLUSIONS:  Despite their common use and far reaching consequences for workers claiming disabling injury or illness, research on the reliability of medical evaluations of disability for work is limited and indicates high variation in judgments among assessing professionals. Standardising the evaluation process could improve reliability. Development and testing of instruments and structured approaches to improve reliability in evaluation of disability are urgently needed.

Expectation of sickness absence duration: a review on statements and methods used in guidelines in Europe and North America.

BACKGROUND: Certifying physicians play a key role in the management of sickness absence and are often provided with guidelines. Some of these guidelines contain statements on expected sickness absence duration, according to diagnosis. We were interested in exploring the evidence base of these statements. METHODS: We identified guidelines through a survey of EUMASS members and a literature search of the Internet and PubMed. We extracted the statements and methods from the guidelines. We compared: diagnoses that were addressed, expected durations and development processes followed. Next, we presented our findings to the developers, to afford them an opportunity to comment and/or correct any misinterpretations. RESULTS: We identified 4 guidelines from social insurance institutions (France, Serbia, Spain and Sweden) and 4 guidelines from private organisations (1 Netherlands, 3 US). Guidelines addressed between 63 and some 63000 health conditions (ICD 10 codes). Health conditions overlapped among guidelines. Direct comparison is hampered by differences in coding (ICD 9 or 10) and level of aggregation (three or four digit, clustering of diseases and treatment situations). Expectations about duration are defined as minimum, maximum, and optimum or mean or median and percentile distribution, stratified to age and work requirements. In a sample of 5 diagnoses we found overlap in expected duration but also differences. Guidelines are developed differently, pragmatic expert consensus being used most, supplemented with data on sickness absence from different registers, other guidelines and non-systematic literature reviews. The effectiveness of these guidelines has not yet been formally evaluated. CONCLUSIONS: Expectations about duration of sickness absence by diagnosis are expressed in several guidelines. The expectations are difficult to compare, their evidence base is unclear and their effectiveness needs to be established.

Sickness certification in primary care: a survey on views and practices among Swiss physicians.

QUESTIONS UNDER STUDY: Studies from several countries (Scandinavia, United Kingdom) report that general practitioners (GPs) experience problems in sickness certification. Our study explored views of Swiss GPs towards sickness certification, their practice and experience, professional skills and problematic interactions with patients. METHODS: We conducted an online survey among GPs throughout Switzerland, exploring behaviour of physicians, patients and employers with regard to sickness certification; GPs' views about sickness certification; required competences for certifying sickness absence, and approaches to advance their competence. We piloted the questionnaire and disseminated it through the networks of the five Swiss academic institutes for primary care. RESULTS: We received 507 valid responses (response rate 50%). Only 43/507 GPs experienced sickness certification as problematic per se, yet 155/507 experienced problems in sickness certification at least once a week. The 507 GPs identified estimating a long-term prognosis about work capacity (64%), handling conflicts with patients (54%), and determining the reduction of work capacity (42%) as problematic. Over 75% would welcome special training opportunities, e.g., on sickness certifications during residency (93%), in insurance medicine (81%), and conflict management (80%). CONCLUSION: Sickness certification as such does not present a major problem to Swiss GPs, which contrasts with the experience in Scandinavian countries and in the UK. Swiss GPs did identify specific tasks of sickness certification as problematic. Training opportunities on sick-leave certification and insurance medicine in general were welcomed.

Evaluation of work capacity in Switzerland: a survey among psychiatrists about practice and problems.

QUESTIONS UNDER STUDY: In Switzerland, psychiatric evaluations of work capacity for determining a person's eligibility for disability benefits are being criticised for a lack of transparency and high inter-rater variability. The aims of this study were to learn about the current practice of psychiatrists, to explore possible sources for lack of transparency and variability, and to contrast practice with current professional guidance. METHODS: A national online-survey among psychiatrists who performed five or more evaluations of work capacity per year. Based on discussions with experts and a literature review, we structured questions focusing on reporting on work capacity, the description of a claimant's previous job, and measures of quality assurance. RESULTS: A total of 129 psychiatrists responded (31% of estimated 412 eligible psychiatrists). The majority reported using instructions of the insurers (77%), peer consulting (65%) and process guidelines (51%). They expressed a claimant's work capacity as free text and percentage work capacity (49%), percentage only (23%), or free text only (14%). A total of 13% used instruments to document work capacity. Psychiatrists considered three different interpretations of percentage work capacity as equally applicable. A job description was regarded as mandatory to determine work capacity by 90% but only 26% received it and found it mostly deficient. CONCLUSIONS: The transparency and reliability of Swiss psychiatrists' conclusions on a claimant's work capacity may be reduced by unsystematic reporting, variable interpretation of the percentage work capacity, lack of a detailed job description and insufficient quality control. Education, engagement of insurers and new guidelines might be effective means of implementing improvements.

Return to work coordination programmes for work disability: a meta-analysis of randomised controlled trials.

BACKGROUND: The dramatic rise in chronically ill patients on permanent disability benefits threatens the sustainability of social security in high-income countries. Social insurance organizations have started to invest in promising, but costly return to work (RTW) coordination programmes. The benefit, however, remains uncertain. We conducted a systematic review to determine the long-term effectiveness of RTW coordination compared to usual practice in patients at risk for long-term disability. METHODS AND FINDINGS: Eligible trials enrolled employees on work absence for at least 4 weeks and randomly assigned them to RTW coordination or to usual practice. We searched 5 databases (to April 2, 2012). Two investigators performed standardised eligibility assessment, study appraisal and data extraction independently and in duplicate. The GRADE framework guided our assessment of confidence in the meta-analytic estimates. We identified 9 trials from 7 countries, 8 focusing on musculoskeletal, and 1 on mental complaints. Most trials followed participants for 12 months or less. No trial assessed permanent disability. Moderate quality evidence suggests a benefit of RTW coordination on proportion at work at end of follow-up (risk ratio = 1.08, 95% CI = 1.03 to 1.13; absolute effect = 5 in 100 additional individuals returning to work, 95% CI = 2 to 8), overall function (mean difference [MD] on a 0 to 100 scale = 5.2, 95% CI = 2.4 to 8.0; minimal important difference [MID] = 10), physical function (MD = 5.3, 95% CI = 1.4 to 9.1; MID = 8.4), mental function (MD = 3.1, 95% CI = 0.7 to 5.6; MID = 7.3) and pain (MD = 6.1, 95% CI = 3.1 to 9.2; MID = 10). CONCLUSIONS: Moderate quality evidence suggests that RTW coordination results in small relative, but likely important absolute benefits in the likelihood of disabled or sick-listed patients returning to work, and associated small improvements in function and pain. Future research should explore whether the limited effects persist, and whether the programmes are cost effective in the long term.

Systematic development of a communication skills training course for physicians performing work disability assessments: from evidence to practice.

BACKGROUND: Physicians require specific communication skills, because the face-to-face contact with their patients is an important source of information. Although physicians who perform work disability assessments attend some communication-related training courses during their professional education, no specialised and evidence-based communication skills training course is available for them. Therefore, the objectives of this study were: 1) to systematically develop a training course aimed at improving the communication skills of physicians during work disability assessment interviews with disability claimants, and 2) to plan an evaluation of the training course. METHODS: A physician-tailored communication skills training course was developed, according to the six steps of the Intervention Mapping protocol. Data were collected from questionnaire studies among physicians and claimants, a focus group study among physicians, a systematic review of the literature, and meetings with various experts. Determinants and performance objectives were formulated. A concept version of the training course was discussed with several experts before the final training course programme was established. The evaluation plan was developed by consulting experts, social insurance physicians, researchers, and policy-makers, and discussing with them the options for evaluation. RESULTS: A two-day post-graduate communication skills training course was developed, aimed at improving professional communication during work disability assessment interviews. Special focus was on active teaching strategies, such as practising the skills in role-play. An adoption and implementation plan was formulated, in which the infrastructure of the educational department of the institute that employs the physicians was utilised. Improvement in the skills and knowledge of the physicians who will participate in the training course will be evaluated in a randomised controlled trial. CONCLUSIONS: The feasibility and practical relevance of the communication skills training course that was developed seem promising. Such a course may be relevant for physicians in many countries who perform work disability assessments. The development of the first training course of this type represents an important advancement in this field.

The assessment of efforts to return to work in the European Union.

BACKGROUND: Assessment of efforts to promote return-to-work (RTW) includes all efforts (vocational and non-vocational) designed to improve the work ability of the sick-listed employee and increase the chance to return to work. Aim of the study was to investigate whether in 13 European countries these RTW efforts are assessed and to compare the procedures by means of six criteria. METHODS: Data were gathered in the taxonomy project of the European Union of Medicine in Assurance and Social Security and by means of an additional questionnaire. RESULTS: In seven countries RTW efforts are subject of the assessment in relation to the application for disability benefits. Description of RTW efforts is a prerequisite in five countries. Guidelines on the assessment of RTW efforts are only available in the Netherlands and no countries report the use of the ICF model. Based on the results of the additional questionnaire, the assessor is a social scientist or a physician. The information used to assess RTW efforts differs, from a report on the RTW process to medical information. A negative outcome of the assessment leads to delay of the application for disability benefits or to application for rehabilitation subsidy. CONCLUSION: RTW efforts are assessed in half of the participating European countries. When compared, the characteristics of the assessment of RTW efforts in the participating European countries show both similarities and differences. This study may facilitate the gathering and exchange of knowledge and experience between countries on the assessment of RTW efforts.

Interviews for the assessment of long-term incapacity for work: a study on adherence to protocols and principles.

BACKGROUND: Assessments for long-term incapacity for work are performed by Social Insurance Physicians (SIPs) who rely on interviews with claimants as an important part of the process. These interviews are susceptible to bias. In the Netherlands three protocols have been developed to conduct these interviews. These protocols are expert- and practice-based. We studied to what extent these protocols are adhered to by practitioners. METHODS: We compared the protocols with one another and with the ICF and the biopsychosocial approach. The protocols describe semi-structured interviews with comparable but not identical topics. All protocols prescribe that the client's opinion on his capacity for work, and his arguments, need to be determined and assessed. We developed a questionnaire to elicit the adherence SIPs have to the protocols, their underlying principles and topics. We conducted a survey among one hundred fifty-five experienced SIPs in the Netherlands. RESULTS: Ninety-eight SIPs responded (64%). All respondents used some form of protocol, either one of the published protocols or their own mix. We found no significant relation between training and the use of a particular protocol. Ninety percent use a semi-structured interview. Ninety-five percent recognise having to verify what the claimant says and eighty-three percent feel the need to establish a good relation (p = 0.019). Twelve topics are basically always addressed by over eighty percent of the respondents. The claimant's opinion of being fit for his own work or other work, and his claim of incapacity and his health arguments for that claim, reach a hundred percent. Description of claimants' previous work reaches ninety-nine percent. CONCLUSION: Our study shows professional consensus among experienced Dutch SIPs about the principle of assessment on arguments, the principle of conducting a semi-structured interview and the most crucial interview topics. This consensus can be used to further develop a protocol for interviewing in the assessment of incapacity for work in social insurance. Such a protocol can improve the quality of the assessments in terms of transparency and reproducibility, as well as by enabling clients to better prepare themselves for the assessments.

Disability assessments: effects of response mode and experience.

PURPOSE: To what extent response mode and experience affect the assessment of disability was investigated. METHOD: An experiment was conducted in which 34 medical doctors (17 inexperienced and 15 experienced) were required to assess disability of a videotaped client. Participants either gave a probability assessment after each piece of information or only after all information had been processed (step-by-step, SBS, or end-of-sequence, EOS). They were furthermore required to indicate how confident they were of their judgement and which information was most important for their judgement. RESULTS: Neither response mode nor experience affected the assessment of disability. Only experienced doctors changed their judgement after seeing the video as compared to their judgement after reading the file. Even though all doctors became more confident after seeing the video, this increase was stronger for experienced than inexperienced doctors. Experienced doctors more often mentioned limitations as the basis for their judgements and, to a lesser degree, client's motivation to return to work than inexperienced doctors. CONCLUSIONS: The results suggest that assessments of disability are largely based on the initial representation that is formed after reading the file. The main pitfall is that the final representation is based on general beliefs rather than on actual client information. For training and support this would mean that doctors should be made aware of the extent to which their assessment is anchored in the case at hand.