Optimizing the Quality of Clinical Data in an Australian Aged Care and Disability Service to Improve Care Delivery and Clinical Outcomes: Protocol for an Agile Lean Six Sigma Study.

BACKGROUND: In Australia, aged care and disability service providers are legally required to maintain comprehensive and accurate clinical documentation to meet regulatory and funding requirements and support safe and high-quality care provision. However, evidence suggests that poor-quality clinical data and documentation are widespread across the sector and can substantially affect clinical decision-making and care delivery and increase business costs. OBJECTIVE: In the Optimizing the Quality of Clinical Data in an Australian Aged Care and Disability Service to Improve Care Delivery and Clinical Outcomes (OPTIMISE) study, we aim to use an Agile Lean Six Sigma framework to identify opportunities for the optimization of clinical documentation processes and clinical information systems, implement and test optimization solutions, and evaluate postoptimization outcomes in a large postacute community-based health service providing aged care and disability services in Western Australia. METHODS: A 3-stage prospective optimization study will be conducted. Stage 1 (baseline [T0]) will measure existing clinical data quality, identify root causes of data quality issues across services, and generate optimization solutions. Stage 2 (optimization) will implement and test changes to clinical documentation processes and information systems using incremental Agile sprints. Stage 3 (evaluation) will evaluate changes in primary and secondary outcomes from T0 to 12 months after optimization. The primary outcome is the data quality measured in terms of defects per unit, defects per million opportunities, and Sigma level. The secondary outcomes are care delivery (direct care time), clinical incidents, business outcomes (cost of quality and workforce productivity), and user satisfaction. Case studies will be analyzed to understand the impact of optimization on clinical outcomes and business processes. RESULTS: As of June 1, 2022, stage 1 commenced with T0 data quality audits conducted to measure current data quality. T0 data quality audits will be followed by user consultations to identify root causes of data quality issues. Optimization solutions will be developed by May 2023 to inform optimization (stage 2) and evaluation (stage 3). Results are expected to be published in June 2023. CONCLUSIONS: The study findings will be of interest to individuals and organizations in the health care sector seeking novel solutions to improve the quality of clinical data, support high-quality care delivery, and reduce business costs. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/39967.

Is Collecting Patient Feedback "a Futile Exercise" in the Context of Recertification?

OBJECTIVE: Patient feedback is considered integral to maintaining excellence, patient safety, and professional development. However, the collection of and reflection on patient feedback may pose unique challenges for psychiatrists. This research uniquely explores the value, relevance, and acceptability of patient feedback in the context of recertification. METHODS: The authors conducted statistical and inductive thematic analyses of psychiatrist responses (n = 1761) to a national census survey of all doctors (n = 26,171) licensed to practice in the UK. Activity theory was also used to develop a theoretical understanding of the issues identified. RESULTS: Psychiatrists rate patient feedback as more useful than some other specialties. However, despite asking a comparable number of patients, psychiatrists receive a significantly lower response rate than most other specialties. Inductive thematic analysis identified six key themes: (1) job role, setting, and environment; (2) reporting issues; (3) administrative barriers; (4) limitations of existing patient feedback tools; (5) attitudes towards patient feedback; and (6) suggested solutions. CONCLUSIONS: The value, relevance, and acceptability of patient feedback are undermined by systemic tensions between division of labor, community understanding, tool complexity, and restrictive rule application. This is not to suggest that patient feedback is "a futile exercise." Rather, existing feedback processes should be refined. In particular, the value and acceptability of patient feedback tools should be explored both from a patient and professional perspective. If issues identified remain unresolved, patient feedback is at risk of becoming a "futile exercise" that is denied the opportunity to enhance patient safety, quality of care, and professional development.

Sociodemographic and educational characteristics of doctors applying for psychiatry training in the UK: secondary analysis of data from the UK Medical Education Database project.

Aims and methodWorkforce shortages in psychiatry are common worldwide. The international literature provides insights into factors influencing decisions to train in psychiatry but is predominately survey based. This national cohort study aimed to identify the characteristics of doctors who were most likely to apply to psychiatry training programmes. The sample comprised doctors who entered UK medical schools in 2007/8 and who made first-time specialty training applications in 2015. The association between application to psychiatry and doctors' sociodemographic and educational characteristics was examined using multivariable logistic regression. RESULTS: Those most likely to apply were White, privately educated older doctors with below average performance at medical school.Clinical implicationsTo reduce workforce shortages, psychiatry must make itself more attractive to all doctors, especially those from underrepresented groups such as state-educated Black and minority ethnic individuals. Otherwise, national policies to widen participation in the study of medicine by such groups may exacerbate the current recruitment crisis.Declaration of interestNone.

The International Landscape of Medical Licensing Examinations: A Typology Derived From a Systematic Review.

BACKGROUND: National licensing examinations (NLEs) are large-scale examinations usually taken by medical doctors close to the point of graduation from medical school. Where NLEs are used, success is usually required to obtain a license for full practice. Approaches to national licensing, and the evidence that supports their use, varies significantly across the globe. This paper aims to develop a typology of NLEs, based on candidacy, to explore the implications of different examination types for workforce planning. METHODS: A systematic review of the published literature and medical licensing body websites, an electronic survey of all medical licensing bodies in highly developed nations, and a survey of medical regulators. RESULTS: The evidence gleaned through this systematic review highlights four approaches to NLEs: where graduating medical students wishing to practice in their national jurisdiction must pass a national licensing exam before they are granted a license to practice; where all prospective doctors, whether from the national jurisdiction or international medical graduates, are required to pass a national licensing exam in order to practice within that jurisdiction; where international medical graduates are required to pass a licensing exam if their qualifications are not acknowledged to be comparable with those students from the national jurisdiction; and where there are no NLEs in operation. This typology facilitates comparison across systems and highlights the implications of different licensing systems for workforce planning. CONCLUSION: The issue of national licensing cannot be viewed in isolation from workforce planning; future research on the efficacy of national licensing systems to drive up standards should be integrated with research on the implications of such systems for the mobility of doctors to cross borders.

Factors associated with junior doctors' decisions to apply for general practice training programmes in the UK: secondary analysis of data from the UKMED project.

BACKGROUND: The UK, like many high-income countries, is experiencing a worsening shortfall of general practitioners (GPs) alongside an increasing demand for their services. At the same time, factors influencing junior doctors' decisions to apply for GP training are only partially understood and research in this area has been hampered by the difficulties in connecting the datasets that map the journey from student to qualified GP. The UK Medical Education Database (UKMED) has been established to ameliorate this problem by linking institutional data across the spectrum of medical education from school to specialty training. Our study aimed to use UKMED to investigate which demographic and educational factors are associated with junior doctors' decisions to apply for GP training. METHODS: Study data, provided by the UKMED Development Group and accessed remotely, contained longitudinal educational and sociodemographic information on all doctors who entered UK medical schools in the 2007/2008 academic year and who made first-time specialty training applications in 2015. We used multivariable logistic regression models to investigate two binary outcomes, namely (1) application to GP training, possibly alongside applications to other specialties, and (2) application solely to GP training. RESULTS: Of 7634 doctors in the sample, 43% applied to GP training possibly alongside other specialities and 26% applied solely to GP training. The odds of applying to GP training were associated with particular demographic factors (being female, non-white or secondary educated in the UK increased the odds of application) and educational factors (non-graduate entry, intercalation and above-median academic performance during medical school all decreased the odds of application). After adjusting for these factors, both the medical school and the foundation school attended were independently associated with the odds of applying to GP training. CONCLUSIONS: Our findings suggest that the supply and demand imbalance in UK primary care might be improved by (1) efforts to attract greater numbers of female, non-white and UK secondary-educated students into medical schools, and (2) targeting resources at medical and foundation schools that deliver doctors likely to fill significant gaps in the workforce. Further research is required to better understand inter-school differences and to develop strategies to improve recruitment of GP trainees.

Comparing Tolerance of Ambiguity in Veterinary and Medical Students.

Current guidelines suggest that educators in both medical and veterinary professions should do more to ensure that students can tolerate ambiguity. Designing curricula to achieve this requires the ability to measure and understand differences in ambiguity tolerance among and within professional groups. Although scales have been developed to measure tolerance of ambiguity in both medical and veterinary professions, no comparative studies have been reported. We compared the tolerance of ambiguity of medical and veterinary students, hypothesizing that veterinary students would have higher tolerance of ambiguity, given the greater patient diversity and less well-established evidence base underpinning practice. We conducted a secondary analysis of questionnaire data from first- to fourth-year medical and veterinary students. Tolerance of ambiguity scores were calculated and compared using the TAMSAD scale (29 items validated for the medical student population), the TAVS scale (27 items validated for the veterinary student population), and a scale comprising the 22 items common to both scales. Using the TAMSAD and TAVS scales, medical students had a significantly higher mean tolerance of ambiguity score than veterinary students (56.1 vs. 54.1, p<.001 and 60.4 vs. 58.5, p=.002, respectively) but no difference was seen when only the 22 shared items were compared (56.1 vs. 57.2, p=.513). The results do not support our hypothesis and highlight that different findings can result when different tools are used. Medical students may have slightly higher tolerance of ambiguity than veterinary students, although this depends on the scale used.

Identifying applicants suitable to a career in nursing: a value-based approach to undergraduate selection.

AIMS: The aim of this study was to complement existing evidence on the suitability of Multiple Mini Interviews as a potential tool for the selection of nursing candidates on to a BSc (Hons) nursing programme. BACKGROUND: This study aimed to trial the Multiple Mini Interview approach to recruitment with a group of first year nursing students (already selected using traditional interviews). DESIGN: Cross-sectional validation study. This paper reports on the evaluation of the participants' detailed scores from the Multiple Mini Interview stations; their original interview scores and their end of year results. METHODS: This study took place in March 2015. Scores from the seven Multiple Mini Interview stations were analysed to show the internal structure, reliability and generalizability of the stations. Original selection scores from interviews and in-course assessment were correlated with the MMI scores and variation by students' age, gender and disability status was explored. RESULTS: Reliability of the Multiple Mini Interview score was moderate (G = 0·52). The Multiple Mini Interview score provided better differentiation between more able students than did the original interview score but neither score was correlated with the module results. Multiple Mini Interview scores were positively associated with students' age but not their gender or disability status. CONCLUSION: The Multiple Mini Interview reported in this study offers a selection process that is based on the values and personal attributes regarded as desirable for a career in nursing and does not necessarily predict academic success. Its moderate reliability indicates the need for further improvement but it is capable of discriminating between candidates and shows little evidence of bias.

Test-retest stability of patient experience items derived from the national GP patient survey.

PURPOSE: The validity and reliability of various items on the GP Patient Survey (GPPS) survey have been reported, however stability of patient responses over time has not been tested. The purpose of this study was to determine the test-retest reliability of the core items from the GPPS. METHODS: Patients who had recently consulted participating GPs in five general practices across the South West England were sent a postal questionnaire comprising of 54 items concerning their experience of their consultation and the care they received from the GP practice. Patients returning the questionnaire within 3 weeks of mail-out were sent a second identical (retest) questionnaire. Stability of responses was assessed by raw agreement rates and Cohen's kappa (for categorical response items) and intraclass correlation coefficients and means (for ordinal response items). RESULTS: 348 of 597 Patients returned a retest questionnaire (58.3 % response rate). In comparison to the test phase, patients responding to the retest phase were older and more likely to have white British ethnicity. Raw agreement rates for the 33 categorical items ranged from 66 to 100 % (mean 88 %) while the kappa coefficients ranged from 0.00 to 1.00 (mean 0.53). Intraclass correlation coefficients for the 21 ordinal items averaged 0.67 (range 0.44-0.77). CONCLUSIONS: Formal testing of items from the national GP patient survey examining patient experience in primary care highlighted their acceptable temporal stability several weeks following a GP consultation.

The impact of large scale licensing examinations in highly developed countries: a systematic review.

BACKGROUND: To investigate the existing evidence base for the validity of large-scale licensing examinations including their impact. METHODS: Systematic review against a validity framework exploring: Embase (Ovid Medline); Medline (EBSCO); PubMed; Wiley Online; ScienceDirect; and PsychINFO from 2005 to April 2015. All papers were included when they discussed national or large regional (State level) examinations for clinical professionals, linked to examinations in early careers or near the point of graduation, and where success was required to subsequently be able to practice. Using a standardized data extraction form, two independent reviewers extracted study characteristics, with the rest of the team resolving any disagreement. A validity framework was used as developed by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education to evaluate each paper's evidence to support or refute the validity of national licensing examinations. RESULTS: 24 published articles provided evidence of validity across the five domains of the validity framework. Most papers (n = 22) provided evidence of national licensing examinations relationships to other variables and their consequential validity. Overall there was evidence that those who do well on earlier or on subsequent examinations also do well on national testing. There is a correlation between NLE performance and some patient outcomes and rates of complaints, but no causal evidence has been established. CONCLUSIONS: The debate around licensure examinations is strong on opinion but weak on validity evidence. This is especially true of the wider claims that licensure examinations improve patient safety and practitioner competence.

Hematological and Biochemistry Profile and Risk Factors Associated with Pulmonary Tuberculosis Patients in Guyana.

Objective. To evaluate the hematological and biochemistry profile of patients with or without HIV-TB at the Georgetown Chest Clinic, Guyana. Methods. An observational, laboratory based study was designed to assess the relationship of PTB and HIV with patients routine biochemical and hematological values. The study was conducted during the period January 2013 to December 2014; a total sample size of 316 patients was enrolled following exclusion and inclusion criteria. Results. Mean age of study population was 40.1 ± 13.8 (95% CI 38.6-41.7) and most were between 40 and 49 age group (27.8%, 95% CI 23.2-33.0). More males were in the study 74.4% (95% CI 69.3-78.8) than females 81% (95% CI 21.1-30.7). 30% (95% CI 25.3-35.3) had a sputum smear grade of 3+ and 62.5% (95% CI 47.0-75.7) showed a CD4 count <200. The study demonstrated significantly low hemoglobin (Hb) 91.7% (95% CI 78.2-97.1), low WBC 27.8% (95% CI 15.8-44.0), high indirect bilirubin 7.4% (95% CI 2.1-23.3), ALT 41.8% (95% CI 28.4-56.7), and AST 72.2% (95% CI 57.3-83.3) among TB-HIV patients. Homelessness RR (relative risk) 2.2 (95% CI 0.48-12.3), smoking RR 1.09 (95% CI 1.01-1.19), and gender (male) RR 1.2 (95% CI 0.61-2.26) were main associated risk factors. Conclusions. There is slight variation among PTB and PTB-HIV coinfected patients in some hematological and biochemistry parameters.

Interpreting multisource feedback: online study of consensus and variation among GP appraisers.

BACKGROUND: GPs collect multisource feedback (MSF) about their professional practice and discuss it at appraisal. Appraisers use such information to identify concerns about a doctor's performance, and to guide the doctor's professional development plan (PDP). AIM: To investigate whether GP appraisers detect variation in doctors' MSF results, and the degree of consensus in appraisers' interpretations of this information. DESIGN AND SETTING: Online study of GP appraisers in north-east England. METHOD: GP appraisers were invited to review eight anonymised doctors' MSF reports, which represented different patterns of scores on the UK General Medical Council's Patient and Colleague Questionnaires. Participants provided a structured assessment of each doctor's report, and recommended actions for their PDP. Appraiser ratings of each report were summarised descriptively. An 'agreement score' was calculated for each appraiser to determine whether their assessments were more lenient than those of other participants. RESULTS: At least one report was assessed by 101/146 appraisers (69%). The pattern of appraisers' ratings suggested that they could detect variation in GPs' MSF results, and recommend reasonable actions for the doctors' PDP. Increasing appraiser age was associated with more favourable interpretations of MSF results. CONCLUSION: Although preliminary, the finding of broad consensus among GP appraisers in their assessment of MSF reports should be reassuring for GPs, appraisers, and employing organisations. However, if older appraisers are more lenient than younger appraisers in their interpretation of MSF and in the actions they suggest to their appraisees as a result, organisations need to consider what steps could be taken to address such differences.

A study to assess the influence of interprofessional point of care simulation training on safety culture in the operating theatre environment of a university teaching hospital.

Interprofessional point of care or in situ simulation is used as a training tool in our operating theatre directorate with the aim of improving crisis behaviours. This study aimed to assess the impact of interprofessional point of care simulation on the safety culture of operating theatres. A validated Safety Attitude Questionnaire was administered to staff members before each simulation scenario and then re-administered to the same staff members after 6-12 months. Pre- and post-training Safety Attitude Questionnaire-Operating Room (SAQ-OR) scores were compared using paired sample t-tests. Analysis revealed a statistically significant perceived improvement in both safety (p < 0.001) and teamwork (p = 0.013) climate scores (components of safety culture) 6-12 months after interprofessional simulation training. A growing body of literature suggests that a positive safety culture is associated with improved patient outcomes. Our study supports the implementation of point of care simulation as a useful intervention to improve safety culture in theatres.

Twelve tips for assessment psychometrics.

It is incumbent on medical schools to show, both to regulatory bodies and to the public at large, that their graduating students are "fit for purpose" as tomorrow's doctors. Since students graduate by virtue of passing assessments, it is vital that schools quality assure their assessment procedures, standards, and outcomes. An important part of this quality assurance process is the appropriate use of psychometric analyses. This begins with development of an empowering, evidence-based culture in which assessment validity can be demonstrated. Preparation prior to an assessment requires the establishment of appropriate rules, test blueprinting and standard setting. When an assessment has been completed, the reporting of test results should consider reliability, assessor, demographic, and long-term analyses across multiple levels, in an integrated way to ensure the information conveyed to all stakeholders is meaningful.

Rising to the challenge: acute stress appraisals and selection centre performance in applicants to postgraduate specialty training in anaesthesia.

The ability to work under pressure is a vital non-technical skill for doctors working in acute medical specialties. Individuals who evaluate potentially stressful situations as challenging rather than threatening may perform better under pressure and be more resilient to stress and burnout. Training programme recruitment processes provide an important opportunity to examine applicants' reactions to acute stress. In the context of multi-station selection centres for recruitment to anaesthesia training programmes, we investigated the factors influencing candidates' pre-station challenge/threat evaluations and the extent to which their evaluations predicted subsequent station performance. Candidates evaluated the perceived stress of upcoming stations using a measure of challenge/threat evaluation-the cognitive appraisal ratio (CAR)-and consented to release their demographic details and station scores. Using regression analyses we determined which candidate and station factors predicted variation in the CAR and whether, after accounting for these factors, the CAR predicted candidate performance in the station. The CAR was affected by the nature of the station and candidate gender, but not age, ethnicity, country of training or clinical experience. Candidates perceived stations involving work related tasks as more threatening. After controlling for candidates' demographic and professional profiles, the CAR significantly predicted station performance: 'challenge' evaluations were associated with better performance, though the effect was weak. Our selection centre model can help recruit prospective anaesthetists who are able to rise to the challenge of performing in stressful situations but results do not support the direct use of challenge/threat data for recruitment decisions.

Web-based textual analysis of free-text patient experience comments from a survey in primary care.

BACKGROUND: Open-ended questions eliciting free-text comments have been widely adopted in surveys of patient experience. Analysis of free text comments can provide deeper or new insight, identify areas for action, and initiate further investigation. Also, they may be a promising way to progress from documentation of patient experience to achieving quality improvement. The usual methods of analyzing free-text comments are known to be time and resource intensive. To efficiently deal with a large amount of free-text, new methods of rapidly summarizing and characterizing the text are being explored. OBJECTIVE: The aim of this study was to investigate the feasibility of using freely available Web-based text processing tools (text clouds, distinctive word extraction, key words in context) for extracting useful information from large amounts of free-text commentary about patient experience, as an alternative to more resource intensive analytic methods. METHODS: We collected free-text responses to a broad, open-ended question on patients' experience of primary care in a cross-sectional postal survey of patients recently consulting doctors in 25 English general practices. We encoded the responses to text files which were then uploaded to three Web-based textual processing tools. The tools we used were two text cloud creators: TagCrowd for unigrams, and Many Eyes for bigrams; and Voyant Tools, a Web-based reading tool that can extract distinctive words and perform Keyword in Context (KWIC) analysis. The association of patients' experience scores with the occurrence of certain words was tested with logistic regression analysis. KWIC analysis was also performed to gain insight into the use of a significant word. RESULTS: In total, 3426 free-text responses were received from 7721 patients (comment rate: 44.4%). The five most frequent words in the patients' comments were "doctor", "appointment", "surgery", "practice", and "time". The three most frequent two-word combinations were "reception staff", "excellent service", and "two weeks". The regression analysis showed that the occurrence of the word "excellent" in the comments was significantly associated with a better patient experience (OR=1.96, 95%CI=1.63-2.34), while "rude" was significantly associated with a worse experience (OR=0.53, 95%CI=0.46-0.60). The KWIC results revealed that 49 of the 78 (63%) occurrences of the word "rude" in the comments were related to receptionists and 17(22%) were related to doctors. CONCLUSIONS: Web-based text processing tools can extract useful information from free-text comments and the output may serve as a springboard for further investigation. Text clouds, distinctive words extraction and KWIC analysis show promise in quick evaluation of unstructured patient feedback. The results are easily understandable, but may require further probing such as KWIC analysis to establish the context. Future research should explore whether more sophisticated methods of textual analysis (eg, sentiment analysis, natural language processing) could add additional levels of understanding.

Prediction of implant loss and marginal bone loss by analysis of dental panoramic radiographs.

PURPOSE: One of the major factors governing implant success is the quantity and density of the host bone. The aim of this work was to determine whether mandibular bone texture and cortical width measurements on plain radiographs could be associated with implant failure and/or marginal bone loss. MATERIALS AND METHODS: A statistical model was built to predict implant failure; it incorporated several radiographic features of cortical and cancellous bone texture, cortical width, and patient smoking habits. Cortical width measurements and texture measurements of cortical and cancellous bone were made on the panoramic radiographs of 460 subjects. These were used to predict implant failure and marginal bone loss after 5 years. Receiver operating characteristic (ROC) curve analysis and area under the curve (AUC) were used to determine the diagnostic accuracy of the variables in predicting implant failure and marginal bone loss. Additionally, for 91 of 460 subjects with periapical radiographs, marginal bone levels around implants were measured over a 5-year period. RESULTS: Of the 460 patients assessed for implant failure, 29 had failed implants (93.7% success rate). The ROC curve built from this model had a sensitivity of 62.1% and specificity of 67.5%. The AUC from the model was 0.690 (95% confidence interval [CI] 0.597 to 0.783). A model was also built to predict marginal bone loss. The ROC curve had 78.6% sensitivity and 74.6% specificity (AUC = 0.880, 95% CI = 0.810 to 0.953). Mandibular cortical width was not a significant predictor of either implant failure or bone loss. CONCLUSION: In a retrospective analysis 5 years after implant placement, features of cancellous and cortical bone of the mandible were significant in predicting implant failure and marginal bone loss in a sample of 460 patients.

Medical student and junior doctors' tolerance of ambiguity: development of a new scale.

The practice of medicine involves inherent ambiguity, arising from limitations of knowledge, diagnostic problems, complexities of treatment and outcome and unpredictability of patient response. Research into doctors' tolerance of ambiguity is hampered by poor conceptual clarity and inadequate measurement scales. We aimed to create and pilot a measurement scale for tolerance of ambiguity in medical students and junior doctors that addresses the limitations of existing scales. After defining tolerance of ambiguity, scale items were generated by literature review and expert consultation. Feedback on the draft scale was sought and incorporated. 411 medical students and 75 foundation doctors in Exeter, UK were asked to complete the scale. Psychometric analysis enabled further scale refinement and comparison of scale scores across subgroups. The pilot study achieved a 64% response rate. The final 29 item version of the Tolerance of Ambiguity in Medical Students and Doctors (TAMSAD) scale had good internal reliability (Cronbach's α 0.80). Tolerance of ambiguity was higher in foundation year 2 doctors than first, third and fourth year medical students (-5.23, P = 0.012; -5.98, P = 0.013; -4.62, P = 0.035, for each year group respectively). The TAMSAD scale offers a valid and reliable alternative to existing scales. Further work is required in different settings and in longitudinal studies but this study offers intriguing provisional insights.

Understanding high and low patient experience scores in primary care: analysis of patients' survey data for general practices and individual doctors.

OBJECTIVES: To determine the extent to which practice level scores mask variation in individual performance between doctors within a practice. DESIGN: Analysis of postal survey of patients' experience of face-to-face consultations with individual general practitioners in a stratified quota sample of primary care practices. SETTING: Twenty five English general practices, selected to include a range of practice scores on doctor-patient communication items in the English national GP Patient Survey. PARTICIPANTS: 7721 of 15,172 patients (response rate 50.9%) who consulted with 105 general practitioners in 25 practices between October 2011 and June 2013. MAIN OUTCOME MEASURE: Score on doctor-patient communication items from post-consultation surveys of patients for each participating general practitioner. The amount of variance in each of six outcomes that was attributable to the practices, to the doctors, and to the patients and other residual sources of variation was calculated using hierarchical linear models. RESULTS: After control for differences in patients' age, sex, ethnicity, and health status, the proportion of variance in communication scores that was due to differences between doctors (6.4%) was considerably more than that due to practices (1.8%). The findings also suggest that higher performing practices usually contain only higher performing doctors. However, lower performing practices may contain doctors with a wide range of communication scores. CONCLUSIONS: Aggregating patients' ratings of doctors' communication skills at practice level can mask considerable variation in the performance of individual doctors, particularly in lower performing practices. Practice level surveys may be better used to "screen" for concerns about performance that require an individual level survey. Higher scoring practices are unlikely to include lower scoring doctors. However, lower scoring practices require further investigation at the level of the individual doctor to distinguish higher and lower scoring general practitioners.

Why do GPs exclude patients from participating in research? An exploration of adherence to and divergence from trial criteria.

BACKGROUND: The role of GPs in recruiting or excluding participants critically underpins the feasibility, external validity and generalizability of primary care research. A better understanding of this role is needed. AIM: To investigate why GPs excluded potentially eligible participants from a large scale randomized controlled trial (RCT), to determine the proportion of patients excluded on account of trial eligibility compared with other reasons, and to explore the impact of such exclusions on the management and generalizability of RCTs. DESIGN AND SETTING: Secondary analysis of data from the CoBalT study, a multi-centre general-practice-based RCT investigating cognitive behavioural therapy as an adjunct to pharmacotherapy for treatment-resistant depression. METHOD: GPs were asked to screen patient lists generated from computerized record searches for trial eligibility and to provide narrative reasons for excluding patients. These reasons were coded independently by two researchers, with a third researcher resolving discrepancies. RESULTS: Thirty-one percent (4750/15,379) of patients were excluded at the GP screening stage, including 663 on patient lists that remained unscreened. Of the 4087 actively excluded patients, 67% were excluded on account of trial exclusion criteria, 20% for other criteria (half of which were comorbid conditions) and 13% without reason. CONCLUSION: Clear, comprehensive criteria, particularly with regards to comorbidities, are required for GPs to confidently screen patients for potential participation in research. Future studies should promote inclusivity and encourage GPs to adopt a liberal approach when screening patient lists. This would enhance the validity and generalizability of primary care research and encourage greater patient autonomy.

GPAQ-R: development and psychometric properties of a version of the general practice assessment questionnaire for use for revalidation by general practitioners in the UK.

BACKGROUND: The General Practice Assessment Questionnaire (GPAQ) has been widely used to assess patient experience in general practice in the UK since 2004. In 2013, new regulations were introduced by the General Medical Council (GMC) requiring UK doctors to undertake periodic revalidation, which includes assessment of patient experience for individual doctors. We describe the development of a new version of GPAQ - GPAQ-R which addresses the GMC's requirements for revalidation as well as additional NHS requirements for surveys that GPs may need to carry out in their own practices. METHODS: Questionnaires were given out by doctors or practice staff after routine consultations in line with the guidance given by the General Medical Council for surveys to be used for revalidation. Data analysis and practice reports were provided independently. RESULTS: Data were analysed for questionnaires from 7258 patients relating to 164 GPs in 29 general practices. Levels of missing data were generally low (typically 4.5-6%). The number of returned questionnaires required to achieve reliability of 0.7 were around 35 for individual doctor communication items and 29 for a composite score based on doctor communication items. This suggests that the responses to GPAQ-R had similar reliability to the GMC's own questionnaire and we recommend 30 completed GPAQ-R questionnaires are sufficient for revalidation purposes. However, where an initial screen raises concern, the survey might be repeated with 50 completed questionnaires in order to increase reliability. CONCLUSIONS: GPAQ-R is a development of a well-established patient experience questionnaire used in general practice in the UK since 2004. This new version can be recommended for use in order to meet the UK General Medical Council's requirements for surveys to be used in revalidation of doctors. It also meets the needs of GPs to ask about patient experience relating to aspects of practice care that are not specific to individual general practitioners (e.g. receptionists, telephone access) which meet other survey requirements of the National Health Service in England. Use of GPAQ-R has the potential to reduce the number of surveys that GPs need to carry out in their practices to meet the various regulatory requirements which they face.