Feedback reports to the general practitioner (GP) on the patients' experiences: are GPs interested, and is this interest associated with GP factors and patient experience scores?

BACKGROUND: Patient experience feedback is key in patient centred health systems, but empirical evidence of general practitioner (GP) interest in it is sparse. We aimed to: (i) quantitatively estimate the level of GP interest for feedback reports on patient experience; (ii) explore determinants of such interest; and (iii) examine potential association between a priori interest and patient experience. METHODS: The patient experience survey included maximum 300 randomly selected patients for each of 50 randomly selected GPs (response rate 41.4%, n = 5,623). GPs were sent a postal letter offering feedback reports and were grouped according to their replies: (i) interested in the report; (ii) not interested. Associations between interest and GP variables were assessed with Chi-square tests and multivariate logistic regression, while associations between interest and scores for 5 patient experiences scales were assessed with multilevel regression models. RESULTS: About half (n = 21; 45.7%) of the GPs showed interest in the report by asking to receive the report. The only GP variable associated with a priori interest was being a specialist in general practice (58.6% vs. 23.5% for those without) (P = 0.021). Interest was significantly associated with the practice patient experience scale (4.1 higher score compared with those not interested, P = 0.048). Interest in the report had small and nonsignificant associations with the remaining patient experience scales. CONCLUSIONS: Almost half of the GPs, and almost 3 in 5 of specialists in general practice, were interested in receiving a GP-specific feedback report on patient experiences. Interest in the report was generally not related to patient experience scores.

Patient experiences with general practitioners: psychometric performance of the generic PEQ-GP instrument among patients with chronic conditions.

BACKGROUND: Most generic patient experience instruments have not been validated specifically for persons with chronic health problems, even though they are the dominant user of GPs/family physicians. OBJECTIVES: To assess the psychometric properties of the generic Patient Experiences with GP Questionnaire (PEQ-GP) instrument (five scales: assessment of GP, coordination, patient enablement, accessibility, and practice) in persons with chronic conditions, and to develop a short version to maximize response rates and minimize respondent fatigue in future applications. METHODS: Secondary analysis of data from a national survey of patient experiences with general practitioners in 2018-2019 (response rate: 42.6%). The psychometric properties of PEQ-GP were assessed with exploratory factor analysis and Cronbach's alpha, supplemented with confirmatory factor analysis (CFA) and item response theory (IRT). A short version was constructed and evaluated based on item performance. RESULTS: Nine hundred and seventy persons reported a chronic condition(s), the most frequent being "musculoskeletal, arthritis, other back and joints" (n = 473, 48.8%). Factor analysis identified three scales with adequate psychometric results: GP (15 items; Cronbach's alpha: 0.96), practice (3 items; Cronbach's alpha: 0.87), and accessibility (2 items; Cronbach's alpha: 0.77). Evaluation of item performance identified a 7-item short version, including a 5-item GP scale with scores with strong concordance with the 15-item scale (Intraclass Correlation Coefficient: 0.97, P < 0.001). CONCLUSIONS: The generic PEQ-GP exhibits adequate psychometric performance for persons with chronic conditions. Three empirically derived PEQ-GP scales cover evaluation of the GP, accessibility, and practice. The 7-item short form minimize respondent burden, but further validation work is warranted before large-scale use.

The association between survey timing and patient-reported experiences with hospitals: results of a national postal survey.

BACKGROUND: Research on the effect of survey timing on patient-reported experiences and patient satisfaction with health services has produced contradictory results. The objective of this study was thus to assess the association between survey timing and patient-reported experiences with hospitals. METHODS: Secondary analyses of a national inpatient experience survey including 63 hospitals in the 5 health regions in Norway during the autumn of 2006. 10,912 (45%) patients answered a postal questionnaire after their discharge from hospital. Non-respondents were sent a reminder after 4 weeks. Multilevel linear regression analysis was used to assess the association between survey timing and patient-reported experiences, both bivariate analysis and multivariate analysis controlling for other predictors of patient experiences. RESULTS: Multivariate multilevel regression analysis revealed that survey time was significantly and negatively related to three of six patient-reported experience scales: doctor services (Beta = -0.424, p< 0.05), information about examinations (Beta = -0.566, p < 0.05) and organization (Beta = -0.528, p < 0.05). Patient age, self-perceived health and type of admission were significantly related to all patient-reported experience scales (better experiences with higher age, better health and routine admission), and all other predictors had at least one significant association with patient-reported experiences. CONCLUSIONS: Survey time was significantly and negatively related to three of the six scales for patient-reported experiences with hospitals. Large differences in survey time across hospitals could be problematic for between-hospital comparisons, implying that survey time should be considered as a potential adjustment factor. More research is needed on this topic, including studies with other population groups, other data collection modes and a longer time span.

The Norwegian EUROPEP questionnaire for patient evaluation of general practice: data quality, reliability and construct validity.

BACKGROUND: The EUROPEP is a widely used international instrument to evaluate general practice care from the perspective of patients, but measurement properties including reliability at the GP level are not sufficiently documented. OBJECTIVE: The objectives of this study were to assess the psychometric properties of the Norwegian version of the EUROPEP and estimate GP-level reliability for scales and items. METHODS: Nine hundred patients consulting nine GPs at a medical centre in Norway were invited to complete the Norwegian EUROPEP at home following a recent consultation. We assessed item missing and ceiling effects and used factor analysis to assess the structure of the 23 items of the EUROPEP. Scales were tested for reliability and construct validity, while reliability at the GP level was tested for items and scales. RESULTS: Five hundred and fifty-seven patients (61.9%) returned the questionnaire. Seven of 23 items had missing responses >10% and 20% had high ceiling effects. Factor analysis identified two groups of questions that formed scales with satisfactory internal consistency reliability and validity. The clinical behaviour scale (12 items) and the organization of care scale (4 items) met the criterion of 0.7 for Cronbach's alpha. The GP-level reliability was >0.7 for both scales, but 9 of 23 items were below the criterion of 0.7. CONCLUSIONS: The study identified two scales in the Norwegian EUROPEP instrument with satisfactory psychometric properties. However, high proportions of item non-response large ceiling effects and low GP-level reliability for several items indicate the need for further instrument refinement.

The Generic Short Patient Experiences Questionnaire (GS-PEQ): identification of core items from a survey in Norway.

BACKGROUND: Questionnaires are commonly used to collect patient, or user, experiences with health care encounters; however, their adaptation to specific target groups limits comparison between groups. We present the construction of a generic questionnaire (maximum of ten questions) for user evaluation across a range of health care services. METHODS: Based on previous testing of six group-specific questionnaires, we first constructed a generic questionnaire with 23 items related to user experiences. All questions included a "not applicable" response option, as well as a follow-up question about the item's importance. Nine user groups from one health trust were surveyed. Seven groups received questionnaires by mail and two by personal distribution. Selection of core questions was based on three criteria: applicability (proportion "not applicable"), importance (mean scores on follow-up questions), and comprehensiveness (content coverage, maximum two items per dimension). RESULTS: 1324 questionnaires were returned providing subsample sizes ranging from 52 to 323. Ten questions were excluded because the proportion of "not applicable" responses exceeded 20% in at least one user group. The number of remaining items was reduced to ten by applying the two other criteria. The final short questionnaire included items on outcome (2), clinician services (2), user involvement (2), incorrect treatment (1), information (1), organisation (1), and accessibility (1). CONCLUSION: The Generic Short Patient Experiences Questionnaire (GS-PEQ) is a short, generic set of questions on user experiences with specialist health care that covers important topics for a range of groups. It can be used alone or with other instruments in quality assessment or in research. The psychometric properties and the relevance of the GS-PEQ in other health care settings and countries need further evaluation.

The General Practitioner Experiences Questionnaire (GPEQ): validity and reliability following the inclusion of new accessibility items.

BACKGROUND: The General Practitioner Experiences Questionnaire (GPEQ) measures community mental health centres from the perspective of GP but lacked accessibility scales and documentation of centre level reliability. OBJECTIVES: To assess the psychometric properties of the GPEQ following the inclusion of four new accessibility items and estimate centre level reliability for all scales and items. METHODS: The design of the study is cross-sectional national survey. The setting of the study is postal survey of GPs in Norway evaluating 80 community mental health centres in the four health regions in Norway during autumn of 2008. Three thousand nine hundred and forty-two GPs were sent a postal questionnaire with the GPEQ and were asked to assess their community mental health centre responsible for general adult psychiatric services. RESULTS: Two thousand two hundred and nine (56.0%) GPs returned a completed questionnaire. Psychometric testing including factor analysis and internal consistency reliability identified seven scales with satisfactory reliability and validity: accessibility (two items, new scale), competence (four items), discharge letter (three items), emergency situations (two items), guidance (three items), referrals (three items, new scale) and workforce situation (four items). All scales met the criterion of 0.7 for Cronbach's alpha. The centre level reliability was >0.8 for all scales, while all items met the criterion of 0.7. CONCLUSIONS: The inclusion of new accessibility items resulted in two new scales in addition to the original five scales in the GPEQ, serving to improve the content validity of the instrument. Centre level reliability was high for all scales and items, giving further support to use GPEQ scores as quality indicators.

International health policy survey in 11 countries: assessment of non-response bias in the Norwegian sample.

BACKGROUND: International health policy surveys are used to compare and evaluate health system performance, but little is known about the effects of non-response. The objective of this study was to assess the effects of non-response in the Norwegian part of the Commonwealth Fund international health policy survey in 2009. METHODS: As part of an international health policy survey in 2009 a cross-sectional survey was conducted in Norway among a representative sample of Norwegian general practitioners. 1,400 randomly selected GPs were sent a postal questionnaire including questions about the Norwegian health care system, the quality of the GPs' own practice and the cooperation with specialist health care. The survey included three postal reminders and a telephone follow-up of postal non-respondents. The main outcome measures were increase in response rate for each reminder, the effects of demographic and practice variables on response, the effects of non-response on survey estimates, and the cost-effectiveness of each reminder. RESULTS: After three postal reminders and one telephone follow-up, the response rate was 59.1%. Statistically significant differences between respondents and non-respondents were found for three variables; group vs. solo practice (p = 0.01), being a specialist or not (p < 0.001) and municipality centrality (least central vs. most central, p = 0.03). However, demographic and practice variables had little association with five outcome variables and the overall survey estimates changed little with additional reminders. In addition, the cost-effectiveness of the final reminders was poor. CONCLUSIONS: The response rate in the Norwegian survey was satisfactory, and the effect of non-response was small indicating adequate representativeness. The cost-effectiveness of the final reminders was poor. The Norwegian findings strengthen the international project, but restrictions in generalizability warrant further study in other countries.

General practitioners' evaluation of community psychiatric services: responsiveness to change of the General Practitioner Experiences Questionnaire (GPEQ).

BACKGROUND: Instruments have been developed to assess professional views of the quality of care but have rarely been tested for responsiveness to change. The objective of this study was to test the responsiveness of the General Practitioner Experiences Questionnaire (GPEQ) for the measurement of Community Mental Health Centres in Norway. METHODS: National surveys were conducted in Norway in 2006 (n = 2,415) and 2008 (n = 2,209) to measure general practitioners' evaluation of community mental health centres. GPs evaluated the centres by means of a postal questionnaire, consisting of questions focused on centre quality and cooperation with GPs. As part of the national surveys 75 GPs in 2006 and 66 GPs in 2008 evaluated Hamar community mental health centre. Between the surveys, several quality improvement initiatives were implemented which were directed at cooperation with and guidance for GPs in Stange municipality, one of eight municipalities in Hamar centre catchment area. The main outcome measures were changes in GPEQ scores from 2006 to 2008 for GPs evaluating Hamar community mental health centre from Stange municipality, and changes in scores for GPs in the other seven municipalities and nationally which were assessed for statistical significance. RESULTS: GPs in Stange municipality rated Hamar community mental health centre significantly better on the guidance scale in 2008 than in 2006; on a 0-100 scale where 100 represents the best possible experiences the score was 26.5 in 2006 and 58.3 in 2008 (p < 0.001). Apart from one item about workforce situation, none of the other scales and items showed significant changes. The control group from the other seven municipalities gave significantly poorer rating for the emergency situation scale, the workforce situation scale and seven items in 2008 than in 2006. The national results showed small differences between 2006 and 2008, even though several scales and items were significantly different. A question about changes in centre performance over the last 2-3 years showed that 82% of GPs from Stange municipality reported that Hamar community mental health centre had improved, compared to only 36% from the other seven municipalities and 40% nationally which was statistically significant. CONCLUSIONS: Following the implementation of an initiative designed to enhance service quality, the GPEQ identified expected changes in the guidance scale for the intervention group, indicating that the instrument is responsive to change. The worsening of services for GPs in the control group evaluating Hamar centre warrants further study.

Rasch analysis of the Psychiatric Out-Patient Experiences Questionnaire (POPEQ).

BACKGROUND: The Psychiatric Out-Patient Experiences Questionnaire (POPEQ) is an 11-item core measure of psychiatric out-patients experiences of the perceived outcome of the treatment, the quality of interaction with the clinician, and the quality of information provision. The POPEQ was found to have evidence for reliability and validity following the application of classical test theory but has not previously been assessed by Rasch analysis. METHODS: Two national postal surveys of psychiatric outpatients took place in Norway in 2004 and 2007. The performance of the POPEQ, including item functioning and differential item functioning, was assessed by Rasch analysis. Principal component analysis of item residuals was used to assess the presence of subdimensions. RESULTS: 6,677 (43.3%) and 11,085 (35.2%) psychiatric out patients responded to the questionnaire in 2004 and 2007, respectively. All items in the scale were retained after the Rasch analysis. The resulting scale had reasonably good fit to the Rasch model. The items performed the same for the two survey years and there was no differential item functioning relating to patient characteristics. Principal component analysis of the residuals confirmed that the measure to a high degree is unidimensional. However, the data also reflects three potential subscales, each relating to one of the three included aspects of health care. CONCLUSIONS: The POPEQ had excellent psychometric properties and Rasch analysis further supported the construct validity of the scale by also identifying the three subdimensions originally included as components in the instrument development. The 11-item instrument is recommended in future research on psychiatric out-patient experiences. Future development may lead to the construction of more precise measures of the three subdomains that the POPEQ is based on.

The association between demographic factors, user reported experiences and user satisfaction: results from three casualty clinics in Norway.

BACKGROUND: User reported experiences and satisfaction are increasingly used as basis for quality indicators in the health sector. However, there is limited understanding of factors associated with user reported experiences and satisfaction with casualty clinics. METHODS: A random sample of 542 patients that had contacted any of three casualty clinics from mid April to mid May 2008 was mailed a questionnaire. A reminder was sent to non-respondents after six weeks. Descriptive statistics for four user reported experiences scales and 20 single items are presented. Multivariate regression analysis was used to assess associations between background variables and user reported experiences, and between user reported experiences and user satisfaction. RESULTS: 225 (41.5%) patients, carers and guardians returned a completed questionnaire. Users reported most positive experiences with the doctor services and the nursing services at the casualty clinics; on a scale from 0 to 100, where 100 is the best possible experience the doctor scale was 82 and the nursing scale 81. Users reported least positive experiences with the organization of the casualty clinic, with a scale score of 65. Self perceived health was associated with user satisfaction, while self perceived health and age were associated with user reported experiences with organization of the clinics. A range of user reported experience domains were related to user satisfaction, after controlling for socio-demographic variables, including experiences with doctor services at the clinics, organization of the clinics, information and self perceived incorrect treatment. CONCLUSIONS: Users report positive experiences with the three casualty clinics, with organization as the aspect with largest improvement potential. The importance of age and health status for users' experiences and satisfaction with casualty clinics was shown, but a range of user reported experiences with the clinics were the most important predictors for user satisfaction.

The association between GP and patient ratings of quality of care at outpatient clinics.

BACKGROUND: GPs and patients are frequently asked to evaluate mental health care, but studies including evaluations from both groups are rare. OBJECTIVE: To assess the association between GPs' and patients' assessment of mental health outpatient clinic in Norway and identify important health care predictors for patient and GP satisfaction with the clinics. METHODS: Two cross-sectional national surveys were carried out: survey of GPs in 2006 and patients in 2007 evaluating outpatient clinics at 69 community mental health centres in Norway. A total of 2009 GPs and 9001 outpatients assessed the clinics by means of a postal questionnaire. Main outcome measures were correlations between GP and patient ratings of the outpatient clinics at the clinic level and health care predictors for patient satisfaction and GP satisfaction with the clinics. RESULTS: Clinic scores for GPs' and patients' assessment of waiting time were moderate to highly correlated (0.65), while clinic scores for GP and patient satisfaction had a lower but significant positive association (0.37). Significant positive correlations between clinic scores for GP and patients ratings were found for 38 of the 48 associations tested. The most important predictors for patient satisfaction with the clinics were interaction with the clinician (beta: 0.23) and being met with politeness and respect at the clinic (beta: 0.19), while the most important predictors for GP satisfaction with the clinics were perceived competence (beta: 0.25), rejection of referrals (beta: -0.17) and waiting time for patients (beta: -0.16). CONCLUSIONS: A consistent positive association between GP and patient ratings at the clinic level was identified. Mental health services aiming at improving GP and patient satisfaction should be sensitive to the fact that the two groups prioritize different health care factors.

Nonresponse bias and cost-effectiveness in a Norwegian survey of family physicians.

Low response rates are a common problem in surveys of family physicians leading to uncertainty about the validity of results. In this study, the authors examined the association between multiple reminders and nonresponse bias, survey estimates and costs in a survey of family physicians in Norway (N = 3,463). After three postal reminders and one telephone follow-up, the response rate was 65.9%. They analyzed differences in nine demographic and practice variables between respondents and nonrespondents, the effect of nonresponse bias on survey estimates, and the cost-effectiveness of each reminder. Statistically significant differences between respondents and nonrespondents were found for six variables. However, demographic and practice variables had little association with the main outcome variables, and the overall survey estimates changed little with additional reminders. In addition, the cost-effectiveness of the final reminders was poor.

Overall patient satisfaction with hospitals: effects of patient-reported experiences and fulfilment of expectations.

BACKGROUND: Patient satisfaction and experiences are important parts of healthcare quality, but patient expectations are seldom included in quality assessments. The objective of this study was to estimate the effects of different predictors of overall patient satisfaction with hospitals, including patient-reported experiences, fulfilment of patient expectations and socio-demographic variables. METHODS: Data were collected using a national patient-experience survey of 63 hospitals in the five health regions in Norway during the autumn of 2006. Postal questionnaires were mailed to 24 141 patients after their discharge from hospital. Non-respondents were sent a reminder after 4 weeks. Multivariate linear regression analysis including multilevel regression was used to assess the predictors of overall patient satisfaction with hospitals. RESULTS: Thirteen variables were significantly associated with overall patient satisfaction: two variables about fulfilment of expectations, eight about patient-reported experiences and three socio-demographic variables. The regression model explained 59% of the variation in overall patient satisfaction. The most important predictor of patient satisfaction with hospitals was patient-reported experiences with the nursing services (ß=0.27, p<0.001), followed by fulfilment of patient expectations (ß=0.21, p<0.001), experiences with doctor services (ß=0.12, p<0.001) and perceived incorrect treatment (ß=-0.12, p<0.001). Multilevel regression analysis confirmed most of the findings, but revealed that age was not a significant predictor of overall patient satisfaction. CONCLUSIONS: The study showed that both fulfilment of expectations and patient-reported experiences are distinct from but related to overall patient satisfaction. The most important predictors for overall patient satisfaction with hospitals are patient-reported experiences and fulfilment of expectations.

Parent experiences questionnaire for outpatient child and adolescent mental health services (PEQ-CAMHS Outpatients): reliability and validity following a national survey.

UNLABELLED: ABSTACT: BACKGROUND: Development and evaluation of the PEQ-CAMHS Outpatients, a parent completed questionnaire to measure experiences of outpatient child and adolescent mental health services (CAMHS) in Norway. METHODS: Literature review, parent interviews, pre-testing and a national survey of 17,080 parents of children who received care at one of the 86 outpatient CAMHS in Norway in 2006. Telephone interviews were conducted with a random sample of non-respondents. Levels of missing data, factor structure, internal consistency and construct validity were assessed. RESULTS: 7,906 (46.0%) parents or primary caregivers responded to the questionnaire. Low levels of missing data suggest that the PEQ-CAMHS is acceptable. The questionnaire includes three scales supported by the results of factor analysis: relationship with health personnel (8 items), information and participation (4 items), and outcome (3 items). Item-total correlations were all above 0.6 and Cronbach's alpha correlations ranged from 0.88-0.94. The results of comparisons of scale scores with several variables relating to global satisfaction, outcome, cooperation, information, involvement and waiting time support the construct validity of the instrument. CONCLUSIONS: The PEQ-CAMHS Outpatients questionnaire includes important aspects of outpatient CAMHS from the perspective of the parent. It has evidence for data quality, internal consistency and validity and is recommended in surveys of parent experiences of these services. Future research should assess test-retest reliability and further tests of construct validity that include clinical data are recommended.

The GPs' Experiences Questionnaire (GPEQ): reliability and validity following a national survey to assess GPs' views of district psychiatric services.

BACKGROUND: The measurement of patient and professional views of quality are important components in the evaluation of health care delivery. OBJECTIVE: To describe the development and evaluation of the GPs' Experiences Questionnaire (GPEQ) for assessing the quality of community mental health clinics in Norway. DESIGN: Literature review, GP interviews, pre-testing of questionnaire items and a cross-sectional national survey. SETTING: Postal survey of GPs in Norway evaluating 73 community mental health clinics in the five health regions in Norway during spring of 2006. SUBJECTS: Three thousand four hundred and sixty-three GPs were sent a postal questionnaire with the GPEQ and were asked to assess their community mental health clinic responsible for general adult psychiatric services. RESULTS: Two thousand one hundred and thirty (61.5%) GPs returned a completed questionnaire. Low levels of missing data suggest that the questionnaire is acceptable. Factor analysis identified five scales: workforce situation (four items), discharge letter (three items), competence (four items), guidance (three items) and emergency situations (two items). All scales met the criterion of 0.7 for Cronbach's alpha and test-retest correlations were 0.72-0.87. The results of validity testing were as hypothesized with scale scores significantly related to knowledge of the community mental health clinic, overall satisfaction, negative experiences with the clinic, waiting time and acceptance of referrals. CONCLUSIONS: The GPEQ is a self-administered questionnaire that includes the most important aspects of the GPs' experience of quality at community mental health clinics. All scales have good evidence for internal consistency, test-retest reliability and validity.