Reliability and validity of the Patient Experiences Questionnaire for Interdisciplinary Treatment for Substance Dependence - Continuous Electronic Measurement (PEQ-ITSD - CEM).

BACKGROUND: Inpatient experiences with interdisciplinary treatment for substance dependence and mental health care are measured using continuous electronic measurements in Norway. Major changes in data collection from cross-sectional surveys to continuous measurements necessitated the revalidation of the instrument. The main purpose of the present study was to determine the psychometric properties of the Patient Experiences Questionnaire for Interdisciplinary Treatment for Substance Dependence - Continuous Electronic Measurement (PEQ-ITSD - CEM). We also aimed to develop a short version of this tool, since completing the original version can be burdensome for some patients. METHODS: The study included adult inpatients (aged ≥ 16 years) who received substance-dependence treatment at 102 different sections in Norway during 2020-2022 (n = 2,850). Factor structure and item performance were assessed. A short version was developed based on the psychometric testing results that included item response theory analysis. RESULTS: The PEQ-ITSD - CEM comprised three empirically based scales with good internal consistency, reliability and validity, which covers treatment and personnel (14 items), milieu (6 items) and outcome (5 items). The results supported a seven-item short version, with three items selected for the treatment and personnel scale, two items for the milieu scale and two items for the outcome scale. CONCLUSIONS: The PEQ-ITSD - CEM can be recommended for future assessments of patient experiences with interdisciplinary treatment for substance dependence in Norway and in other countries with similar healthcare systems. This short-form version can be applied when respondent burden is a crucial issue.

Feedback reports to the general practitioner (GP) on the patients' experiences: are GPs interested, and is this interest associated with GP factors and patient experience scores?

BACKGROUND: Patient experience feedback is key in patient centred health systems, but empirical evidence of general practitioner (GP) interest in it is sparse. We aimed to: (i) quantitatively estimate the level of GP interest for feedback reports on patient experience; (ii) explore determinants of such interest; and (iii) examine potential association between a priori interest and patient experience. METHODS: The patient experience survey included maximum 300 randomly selected patients for each of 50 randomly selected GPs (response rate 41.4%, n = 5,623). GPs were sent a postal letter offering feedback reports and were grouped according to their replies: (i) interested in the report; (ii) not interested. Associations between interest and GP variables were assessed with Chi-square tests and multivariate logistic regression, while associations between interest and scores for 5 patient experiences scales were assessed with multilevel regression models. RESULTS: About half (n = 21; 45.7%) of the GPs showed interest in the report by asking to receive the report. The only GP variable associated with a priori interest was being a specialist in general practice (58.6% vs. 23.5% for those without) (P = 0.021). Interest was significantly associated with the practice patient experience scale (4.1 higher score compared with those not interested, P = 0.048). Interest in the report had small and nonsignificant associations with the remaining patient experience scales. CONCLUSIONS: Almost half of the GPs, and almost 3 in 5 of specialists in general practice, were interested in receiving a GP-specific feedback report on patient experiences. Interest in the report was generally not related to patient experience scores.

Patient experiences with general practitioners: psychometric performance of the generic PEQ-GP instrument among patients with chronic conditions.

BACKGROUND: Most generic patient experience instruments have not been validated specifically for persons with chronic health problems, even though they are the dominant user of GPs/family physicians. OBJECTIVES: To assess the psychometric properties of the generic Patient Experiences with GP Questionnaire (PEQ-GP) instrument (five scales: assessment of GP, coordination, patient enablement, accessibility, and practice) in persons with chronic conditions, and to develop a short version to maximize response rates and minimize respondent fatigue in future applications. METHODS: Secondary analysis of data from a national survey of patient experiences with general practitioners in 2018-2019 (response rate: 42.6%). The psychometric properties of PEQ-GP were assessed with exploratory factor analysis and Cronbach's alpha, supplemented with confirmatory factor analysis (CFA) and item response theory (IRT). A short version was constructed and evaluated based on item performance. RESULTS: Nine hundred and seventy persons reported a chronic condition(s), the most frequent being "musculoskeletal, arthritis, other back and joints" (n = 473, 48.8%). Factor analysis identified three scales with adequate psychometric results: GP (15 items; Cronbach's alpha: 0.96), practice (3 items; Cronbach's alpha: 0.87), and accessibility (2 items; Cronbach's alpha: 0.77). Evaluation of item performance identified a 7-item short version, including a 5-item GP scale with scores with strong concordance with the 15-item scale (Intraclass Correlation Coefficient: 0.97, P < 0.001). CONCLUSIONS: The generic PEQ-GP exhibits adequate psychometric performance for persons with chronic conditions. Three empirically derived PEQ-GP scales cover evaluation of the GP, accessibility, and practice. The 7-item short form minimize respondent burden, but further validation work is warranted before large-scale use.

Reliability and validity of the Psychiatric Inpatient Patient Experience Questionnaire - Continuous Electronic Measurement (PIPEQ-CEM).

BACKGROUND: The increasing emphasis on patient-centred care has accelerated the demand for high-quality assessment instruments, but the development and application of measures of the quality of care provided for mental health have lagged behind other areas of medicine. The main objective of this study was to determine the psychometric properties of the Psychiatric Inpatient Patient Experience Questionnaire - Continuous Electronic Measurement (PIPEQ-CEM), which consists of large-scale measurements from a Norwegian population. The change from cross-sectional surveys to continuous measurements necessitated further validation of the instrument. The secondary objective was to develop a short version of the PIPEQ-CEM. METHODS: The data included responses from the first year of continuous measurement, and included adult inpatients (age ≥ 18 years) who received specialized mental healthcare from 191 different sections in Norway (n = 3,249). Missing data, ceiling effects, factor structure and internal consistency levels were assessed. The short scale was developed by exploring missing items, ceiling effects, results from exploratory factor analysis (EFA) and item performance from item response theory (IRT) analyses. RESULTS: Psychometric testing supported previous results and illustrated that the PIPEQ-CEM comprises three empirically based scales with good internal consistency, reliability and validity, and covers structure and facilities, patient-centred interactions, and outcomes. A seven-item short form was developed, which provides an efficient approach for brief yet comprehensive measurements that can be applied in the future. CONCLUSION: The PIPEQ-CEM can be recommended for use in future national surveys that assess patient experience with inpatient psychiatric care in Norway and in other countries with similar healthcare systems. The short form can be applied where respondent burden and cognitive load are crucial issues.

Are parents' geographical origin associated with their evaluation of child and adolescent mental health services? Results from a national survey in Norway.

The objective was to assess the association between parents' geographical origin and their evaluation of outpatient child and adolescent mental health services (CAMHS). Data were collected in a national parent's experience survey of all outpatient CAMHS in Norway in 2017. Following exclusions, 16,143 parents were part of the study, of which 5932 responded (36.1%). Diagnosis and global psychosocial functional level were collected from the National Patient Register. Multilevel regression was used to assess the association between parents' geographical origin and parent evaluation of the outpatient CAMHS on ten indicators. Sentiment and content analysis was conducted on open-ended comments from parents. The estimated regression coefficients showed that parents born in Eastern Europe scored the services significantly poorer than parents born in Norway on outcome of treatment (- 7.73, p < 0.01), general satisfaction (- 5.53, p < 0.05), ease of getting in contact with health personnel outside of scheduled appointments (- 17.04, p < 0.001), and knowledge of the services that the child has received at the service (- 10.63, p < 0.001). Parents born in Asia/Africa/South America scored the services similar as Norwegian parents on eight of ten indicators, better on one (waiting time) and poorer on one (ease of getting in contact). Sentiment analysis showed that 54% of the comments from parents born in Eastern Europe were negative, compared to 42% for the Norwegian group and 36% for Asia/Africa/South America. The parents' evaluation of the outpatient CAMHS were partly associated with their geographical origin, with parents born in Eastern Europe reporting poorer experiences than parents born in Norway.

Patient experiences with general practice in Norway: a comparison of immigrant groups and the majority population following a national survey.

BACKGROUND: Patient experience is an important indicator of quality of health care. In Norway, little is known about the quality of health care for immigrants. The aim of this study was to compare patient-reported experiences with general practice between the Norwegian-born population and immigrant groups. METHODS: We performed secondary analyses of data from a national survey on patient experiences with general practice, including assessments of general practitioners (GPs) and their GP offices. The survey was carried out in Norway in 2018-19. The total number of respondents was 2029, with a response rate of 42.6%. Region of birth was available for 1981 participants, and these were included in the analyses ("Norway" (N = 1756), "Asia, Africa or South America" (N = 95), "Eastern Europe" (N = 70) and "Western Europe, North America or Oceania" (N = 60)). Five indicators of patient experiences were used as dependent variables in bivariate and multivariate analyses, with region of birth as the main exposure variable and other background variables about the patient as adjustment variables: "the GP" (measures related to communication and competency), "auxiliary staff" (politeness, competency, organization), "accessibility" (waiting times), "coordination" (with other services) and `enablement` (GP facilitates coping with/understanding illness). RESULTS: Immigrants as a whole reported poorer experiences with general practice than the majority population, with significantly poorer scores on four of five patient experience indicators. Patients from Asia/Africa/South America reported poorer experiences than those from Norway on the indicators "GP", "auxiliary staff", "accessibility" and "coordination": on a scale from 0 to 100 where 100 is the best, the difference ranged from 7.8 (GP) to 20.3 (accessibility). Patients from Eastern Europe reported lower scores on "GP" and patients from Western Europe/North America/Oceania reported lower scores on "auxiliary staff". These associations were still significant after adjustment for sex, age, self-rated physical and mental health, number of contacts with the GP and education. CONCLUSIONS: For countries with a substantial proportion of foreign-born patients in the health system, immigrant background is an important parameter in quality improvement work. Immigrant background is also an important parameter in health service research.

A randomized comparison of three data collection models for the measurement of parent experiences with diabetes outpatient care.

BACKGROUND: The objective of this study was to compare three data collection methods for the measurement of parent experiences with hospital outpatient care for child and adolescent diabetes, based on a randomised national trial in Norway involving both pen-and-paper and electronic response options. METHODS: The sample frame was patients registered in the Norwegian Childhood Diabetes Registry. Parents of patients were randomised into the following groups (n = 2606): group A, who were posted questionnaires with only a pen-and-paper response option (n = 859); group B, who were posted questionnaires with both an electronic and a pen-and-paper response option (n = 886); and group C, who were posted questionnaires with only an electronic response option (n = 861). The three groups were compared on response rate, background variables about respondents, main study results and survey costs. Statistical analysis included logistic regression to test group differences in response probabilities and multilevel linear regression to test differences in parent experiences. RESULTS: The response rate was 61.8% for group A, 62.4% for group B and 41.6% for group C. The probability of answering was significantly higher for group A (OR = 2.3, p < 0.001) and B (OR = 2.3, p < 0.001) compared to group C. Respondent age, gender, education, living with the child and the degree of participation in consultations did not differ significantly between the three groups. Group differences in parent-reported experiences were small, varying from 1.0 (equipment and doctor contact) to 2.4 (outcome), on a scale from 0 to 100. Only one of 18 group differences was significant: the mixed group had significantly higher score than the electronic group on the organization scale (p < 0.05). The total cost of the electronic model was less than half the cost of the other models, and cost per response was 5.1 euros for the electronic model compared to 8.2 euros for group A and 7.6 euros for group B. CONCLUSIONS: The models with pen-and paper questionnaire included had more than 20% higher response rate than the model with an electronic-only response option. Background variables and parent-reported experiences were similar between the three groups, and the electronic model was the more cost-effective model.

Parent experiences of diabetes care questionnaire (PEQ-DC): reliability and validity following a national survey in Norway.

BACKGROUND: Patient experiences are acknowledged as an important aspect of health care quality but no validated instruments have been identified for the measurement of either parent or patient experiences with outpatient paediatric diabetes care. The aim of the current study was to assess the psychometric properties of a new instrument developed to measure parent experiences of paediatric diabetes care at hospital outpatient departments in Norway. METHODS: The development of the questionnaire was based on a literature review of existing questionnaires, qualitative interviews with both parents and children/adolescents, expert-group consultations, pretesting of the questionnaire and a pilot study. The national pilot study included parents of 2606 children/adolescents aged 0-17 years with Type 1 Diabetes registered in The Norwegian Childhood Diabetes Registry, a nationwide, population-based registry. Levels of missing data, ceiling effects, factor structure, internal consistency, item discriminant validity and construct validity were assessed. RESULTS: A total of 2606 patients were included in the survey, but 80 were excluded due to incorrect addresses. 1399 (55%) parents responded to the questionnaire. Low levels of missing or "not applicable" responses were found for 31 of the 35 items (< 10%), and 27 of 35 items were below the ceiling-effect criterion. Psychometric testing and theoretical considerations identified six scales: Consultation (six items), organisation (five items), equipment (three items), nurse contact (four items), doctor contact (four items) and outcome (five items). All six scales met the 0.7 criterion for Cronbach's alpha (range: 0.71-0.90). As expected, each item had a higher correlation with its hypothesised scale than with any of the other five scales. The construct validity of the Parent Experiences of Diabetes Care Questionnaire (PEQ-DC) was supported by 17 out of 18 associations with variables expected to be related to parent experiences. CONCLUSION: The psychometric testing of the PEQ-DC showed good evidence for data quality, internal consistency and construct validity. The instrument includes important aspects of diabetes care at paediatric outpatient departments from the perspective of the parent. The content validity of the PEQ-DC was secured by a rigorous development process, and the instrument was tested following a national survey in Norway, securing generalisability across Norway.

Patient experiences questionnaire for interdisciplinary treatment for substance dependence (PEQ-ITSD): reliability and validity following a national survey in Norway.

BACKGROUND: Patient experiences are an important aspect of health care quality, but there is a lack of validated instruments for their measurement in the substance dependence literature. A new questionnaire to measure inpatients' experiences of interdisciplinary treatment for substance dependence has been developed in Norway. The aim of this study was to psychometrically test the new questionnaire, using data from a national survey in 2013. METHODS: The questionnaire was developed based on a literature review, qualitative interviews with patients, expert group discussions and pretesting. Data were collected in a national survey covering all residential facilities with inpatients in treatment for substance dependence in 2013. Data quality and psychometric properties were assessed, including ceiling effects, item missing, exploratory factor analysis, and tests of internal consistency reliability, test-retest reliability and construct validity. RESULTS: The sample included 978 inpatients present at 98 residential institutions. After correcting for excluded patients (n = 175), the response rate was 91.4%. 28 out of 33 items had less than 20.5% of missing data or replies in the "not applicable" category. All but one item met the ceiling effect criterion of less than 50.0% of the responses in the most favorable category. Exploratory factor analysis resulted in three scales: "treatment and personnel", "milieu" and "outcome". All scales showed satisfactory internal consistency reliability (Cronbach's alpha ranged from 0.75-0.91) and test-retest reliability (ICC ranged from 0.82-0.85). 17 of 18 significant associations between single variables and the scales supported construct validity of the PEQ-ITSD. CONCLUSION: The content validity of the PEQ-ITSD was secured by a literature review, consultations with an expert group and qualitative interviews with patients. The PEQ-ITSD was used in a national survey in Norway in 2013 and psychometric testing showed that the instrument had satisfactory internal consistency reliability and construct validity.

PIPEQ-OS--an instrument for on-site measurements of the experiences of inpatients at psychiatric institutions.

BACKGROUND: The Psychiatric Inpatient Patient Experience Questionnaire (PIPEQ) was developed for post-discharge measurements of experiences, but the low response rates associated with post-discharge surveys restrict their usefulness. A new questionnaire was developed based on the PIPEQ for on-site measurements of patient experiences: the PIPEQ-OS. The aim of this study was to psychometrically test the PIPEQ-OS using data from a nationally representative survey conducted in Norway in 2014. METHODS: Data were collected using a nationally representative patient-experience survey; 25% of the institutions in each of the 4 health regions in Norway were randomly selected, yielding a total of 26 institutions. The PIPEQ-OS questionnaire was completed by patients on-site on an agreed day in week 37 of 2014. Item missing and ceiling effects were assessed, and factor analysis was used to assess the structure of the items included in the PIPEQ-OS. The scales were tested for internal consistency reliability, test-retest reliability and construct validity. RESULTS: The initial sample comprised 857 patients. Of these, 60 were excluded for ethical reasons and 57 were excluded because they were absent on the day of the survey. Of the remaining 740 patients, 552 (74.6% of the included population) returned the questionnaire. Low levels of missing or "not applicable" responses were found for 18 of the 21 items (<20%), and 20 of 21 items were below the ceiling-effect criterion. Psychometric testing identified three scales: structure and facilities (six items), patient-centred interaction (six items) and outcomes (five items). All scales met the criterion of 0.7 for Cronbach's alpha (range: 0.79-0.91) and test-retest reliability (range: 0.83-0.84). The construct validity of the scales was supported by 14 of 15 significant associations with variables known to be related to psychiatric inpatient experiences. CONCLUSIONS: The PIPEQ-OS comprises three scales with satisfactory internal consistency reliability and construct validity. This instrument can be used for on-site assessments of psychiatric inpatient patient experiences, but further research is needed to evaluate its usefulness as basis for external quality indicators.

The association between patient-reported incidents in hospitals and estimated rates of patient harm.

OBJECTIVE: The aim of this study was to test the association between the rates of patient-reported incidents and patient harm documented in the patient record. DESIGN: The study was a secondary analysis of two national hospital assessments conducted in 2011. SETTING: Hospital services in Norway. PARTICIPANTS: The patient survey was a standard national patient-experience survey conducted at the hospital level for all 63 hospitals in Norway. The medical record review was performed by 47 Global Trigger Tools (GTTs) in all 19 hospital trusts and 4 private hospitals. The two data sets were matched at the unit level, yielding comparable patient experiences and GTT data for 7 departments, 16 hospitals and 11 hospital trusts. INTERVENTION: No intervention. MAIN OUTCOME MEASURES: The correlation at the unit level between the patient-reported incident in hospital instrument (PRIH-I) and estimated rates of patient harm from the GTT. RESULTS: The PRIH-I index was significantly correlated with all patient-reported experience indicators at the individual level, with estimates for all patient harm events (Categories E-I) at the unit level (r = 0.62, P < 0.01), and with estimates of more serious harm events in Categories F-I (r = 0.42, P < 0.05). CONCLUSIONS: Patient-reported incidents in hospitals, as measured by the PRIH-I, are strongly correlated with patient harm rates based on the GTT. This indicates that patient-reported incidents are related to patient safety, but more research is needed to confirm the usefulness of patient reporting in the evaluation of patient safety.

Patient-reported experiences with hospitals: comparison of proxy and patient scores using propensity-score matching.

OBJECTIVE: The objective of this study was to compare patient-experience scores between patients with a proxy response and without a proxy response, using propensity-score matching to maximize the comparability between these two groups. DESIGN: Cross-sectional survey. SETTING: Hospital inpatient services in Norway. PARTICIPANTS: Patients were randomly selected from each of the 61 hospitals in Norway during spring 2011. Postal questionnaires were mailed to 23 420 patients after their discharge from hospital. INTERVENTION: No intervention. MAIN OUTCOME MEASURES: All of the patient respondents (n = 8744) and the matched cases (n = 734) were compared with patients with a proxy response (n = 734) on 12 patient-experience indicators using t-tests. RESULTS: Compared with patient respondents, patients with a proxy response had poorer health, were older and were more often discharged from the hospital to another health-care institution (P < 0.001). Patients with a proxy response yielded significantly lower patient-experience scores than those without a proxy response for 9 of the 12 indicators. Compared with the matched patient group, patients with a proxy response had significantly lower scores for 3 of the 12 indicators and a significantly higher score for one indicator. Differences in scores between patients with a proxy response and the matched patient sample were small, with the largest difference being 4 on a scale of 0-100. CONCLUSIONS: Patients with a proxy response report somewhat poorer experiences than those without a proxy response; however, proxies represent a different patient group than the patient group as a whole, and the results were very similar after controlling for these differences.

The use of data from national and other large-scale user experience surveys in local quality work: a systematic review.

PURPOSE: An important goal for national and large-scale surveys of user experiences is quality improvement. However, large-scale surveys are normally conducted by a professional external surveyor, creating an institutionalized division between the measurement of user experiences and the quality work that is performed locally. The aim of this study was to identify and describe scientific studies related to the use of national and large-scale surveys of user experiences in local quality work. DATA SOURCES: Ovid EMBASE, Ovid MEDLINE, Ovid PsycINFO and the Cochrane Database of Systematic Reviews. STUDY SELECTION: Scientific publications about user experiences and satisfaction about the extent to which data from national and other large-scale user experience surveys are used for local quality work in the health services. DATA EXTRACTION: Themes of interest were identified and a narrative analysis was undertaken. RESULTS OF DATA SYNTHESIS: Thirteen publications were included, all differed substantially in several characteristics. The results show that large-scale surveys of user experiences are used in local quality work. The types of follow-up activity varied considerably from conducting a follow-up analysis of user experience survey data to information sharing and more-systematic efforts to use the data as a basis for improving the quality of care. CONCLUSION: This review shows that large-scale surveys of user experiences are used in local quality work. However, there is a need for more, better and standardized research in this field. The considerable variation in follow-up activities points to the need for systematic guidance on how to use data in local quality work.

Classification of patients based on their evaluation of hospital outcomes: cluster analysis following a national survey in Norway.

BACKGROUND: A general trend towards positive patient-reported evaluations of hospitals could be taken as a sign that most patients form a homogeneous, reasonably pleased group, and consequently that there is little need for quality improvement. The objective of this study was to explore this assumption by identifying and statistically validating clusters of patients based on their evaluation of outcomes related to overall satisfaction, malpractice and benefit of treatment. METHODS: Data were collected using a national patient-experience survey of 61 hospitals in the 4 health regions in Norway during spring 2011. Postal questionnaires were mailed to 23,420 patients after their discharge from hospital. Cluster analysis was performed to identify response clusters of patients, based on their responses to single items about overall patient satisfaction, benefit of treatment and perception of malpractice. RESULTS: Cluster analysis identified six response groups, including one cluster with systematically poorer evaluation across outcomes (18.5% of patients) and one small outlier group (5.3%) with very poor scores across all outcomes. One-Way ANOVA with post-hoc tests showed that most differences between the six response groups on the three outcome items were significant. The response groups were significantly associated with nine patient-experience indicators (p < 0.001), and all groups were significantly different from each of the other groups on a majority of the patient-experience indicators. Clusters were significantly associated with age, education, self-perceived health, gender, and the degree to write open comments in the questionnaire. CONCLUSIONS: The study identified five response clusters with distinct patient-reported outcome scores, in addition to a heterogeneous outlier group with very poor scores across all outcomes. The outlier group and the cluster with systematically poorer evaluation across outcomes comprised almost one-quarter of all patients, clearly demonstrating the need to tailor quality initiatives and improve patient-perceived quality in hospitals. More research on patient clustering in patient evaluation is needed, as well as standardization of methodology to increase comparability across studies.

Web-Based Public Ratings of General Practitioners in Norway: Validation Study.

BACKGROUND: Understanding the complex relationships among multiple strategies for gathering users' perspectives in the evaluation of the performance of services is crucial for the interpretation of user-reported measures. OBJECTIVE: The main objectives were to (1) evaluate the psychometric performance of an 11-item web-based questionnaire of ratings of general practitioners (GPs) currently used in Norway (Legelisten.no) and (2) assess the association between web-based and survey-based patient experience indicators. METHODS: We included all published ratings on GPs and practices on Legelisten.no in the period of May 5, 2012, to December 15, 2021 (N=76,521). The questionnaire consists of 1 mandatory item and 10 voluntary items with 5 response categories (1 to 5 stars), alongside an open-ended review question and background variables. Questionnaire dimensionality and internal consistency were assessed with Cronbach α, exploratory factor, and item response theory analyses, and a priori hypotheses were developed for assessing construct validity (chi-square analysis). We calculated Spearman correlations between web-based ratings and reference patient experience indicators based on survey data using the patient experiences with the GP questionnaire (n=5623 respondents for a random sample of 50 GPs). RESULTS: Web-based raters were predominantly women (n=32,074, 64.0%), in the age range of 20-50 years (n=35,113, 74.6%), and reporting 5 or fewer consultations with the GP each year (n=28,798, 64.5%). Ratings were missing for 18.9% (n=14,500) to 27.4% (n=20,960) of nonmandatory items. A total of 4 of 11 rating items showed a U-shaped distribution, with >60% reporting 5 stars. Factor analysis and internal consistency testing identified 2 rating scales: "GP" (5 items; α=.98) and "practice" (6 items; α=.85). Some associations were not consistent with a priori hypotheses and allowed only partial confirmation of the construct validity of ratings. Item response theory analysis results were adequate for the "practice" scale but not for the "GP" scale, with items with inflated discrimination (>5) distributed over a narrow interval of the scale. The correlations between the web-based ratings GP scale and GP reference indicators ranged from 0.34 (P=.021) to 0.44 (P=.002), while the correlation between the web-based ratings practice scale and reference indicators ranged from 0.17 (not significant) to 0.49 (P<.001). The strongest correlations between web-based and survey scores were found for items measuring practice-related experiences: phone availability (ρ=0.51), waiting time in the office (ρ=0.62), other staff (ρ=0.54-0.58; P<.001). CONCLUSIONS: The practice scale of the web-based ratings has adequate psychometric performance, while the GP suffers from important limitations. The associations with survey-based patient experience indicators were accordingly mostly weak to modest. Our study underlines the importance of interpreting web-based ratings with caution and the need to further develop rating sites.

The association between survey timing and patient-reported experiences with hospitals: results of a national postal survey.

BACKGROUND: Research on the effect of survey timing on patient-reported experiences and patient satisfaction with health services has produced contradictory results. The objective of this study was thus to assess the association between survey timing and patient-reported experiences with hospitals. METHODS: Secondary analyses of a national inpatient experience survey including 63 hospitals in the 5 health regions in Norway during the autumn of 2006. 10,912 (45%) patients answered a postal questionnaire after their discharge from hospital. Non-respondents were sent a reminder after 4 weeks. Multilevel linear regression analysis was used to assess the association between survey timing and patient-reported experiences, both bivariate analysis and multivariate analysis controlling for other predictors of patient experiences. RESULTS: Multivariate multilevel regression analysis revealed that survey time was significantly and negatively related to three of six patient-reported experience scales: doctor services (Beta = -0.424, p< 0.05), information about examinations (Beta = -0.566, p < 0.05) and organization (Beta = -0.528, p < 0.05). Patient age, self-perceived health and type of admission were significantly related to all patient-reported experience scales (better experiences with higher age, better health and routine admission), and all other predictors had at least one significant association with patient-reported experiences. CONCLUSIONS: Survey time was significantly and negatively related to three of the six scales for patient-reported experiences with hospitals. Large differences in survey time across hospitals could be problematic for between-hospital comparisons, implying that survey time should be considered as a potential adjustment factor. More research is needed on this topic, including studies with other population groups, other data collection modes and a longer time span.

User-experience surveys with maternity services: a randomized comparison of two data collection models.

OBJECTIVE: To compare two ways of combining postal and electronic data collection for a maternity services user-experience survey. DESIGN: Cross-sectional survey. SETTING: Maternity services in Norway. PARTICIPANTS: All women who gave birth at a university hospital in Norway between 1 June and 27 July 2010. INTERVENTION: Patients were randomized into the following groups (n= 752): Group A, who were posted questionnaires with both electronic and paper response options for both the initial and reminder postal requests; and Group B, who were posted questionnaires with an electronic response option for the initial request, and both electronic and paper response options for the reminder postal request. MAIN OUTCOME MEASURES: Response rate, the amount of difference in background variables between respondents and non-respondents, main study results and estimated cost-effectiveness. RESULTS: The final response rate was significantly higher in Group A (51.9%) than Group B (41.1%). None of the background variables differed significantly between the respondents and non-respondents in Group A, while two variables differed significantly between the respondents and non-respondents in Group B. None of the 11 user-experience scales differed significantly between Groups A and B. The estimated costs per response for the forthcoming national survey was €11.7 for data collection Model A and €9.0 for Model B. CONCLUSIONS: The model with electronic-only response option in the first request had lowest response rate. However, this model performed equal to the other model on non-response bias and better on estimated cost-effectiveness, and is the better of the two models in large-scale user experiences surveys with maternity services.

The Norwegian EUROPEP questionnaire for patient evaluation of general practice: data quality, reliability and construct validity.

BACKGROUND: The EUROPEP is a widely used international instrument to evaluate general practice care from the perspective of patients, but measurement properties including reliability at the GP level are not sufficiently documented. OBJECTIVE: The objectives of this study were to assess the psychometric properties of the Norwegian version of the EUROPEP and estimate GP-level reliability for scales and items. METHODS: Nine hundred patients consulting nine GPs at a medical centre in Norway were invited to complete the Norwegian EUROPEP at home following a recent consultation. We assessed item missing and ceiling effects and used factor analysis to assess the structure of the 23 items of the EUROPEP. Scales were tested for reliability and construct validity, while reliability at the GP level was tested for items and scales. RESULTS: Five hundred and fifty-seven patients (61.9%) returned the questionnaire. Seven of 23 items had missing responses >10% and 20% had high ceiling effects. Factor analysis identified two groups of questions that formed scales with satisfactory internal consistency reliability and validity. The clinical behaviour scale (12 items) and the organization of care scale (4 items) met the criterion of 0.7 for Cronbach's alpha. The GP-level reliability was >0.7 for both scales, but 9 of 23 items were below the criterion of 0.7. CONCLUSIONS: The study identified two scales in the Norwegian EUROPEP instrument with satisfactory psychometric properties. However, high proportions of item non-response large ceiling effects and low GP-level reliability for several items indicate the need for further instrument refinement.

The Generic Short Patient Experiences Questionnaire (GS-PEQ): identification of core items from a survey in Norway.

BACKGROUND: Questionnaires are commonly used to collect patient, or user, experiences with health care encounters; however, their adaptation to specific target groups limits comparison between groups. We present the construction of a generic questionnaire (maximum of ten questions) for user evaluation across a range of health care services. METHODS: Based on previous testing of six group-specific questionnaires, we first constructed a generic questionnaire with 23 items related to user experiences. All questions included a "not applicable" response option, as well as a follow-up question about the item's importance. Nine user groups from one health trust were surveyed. Seven groups received questionnaires by mail and two by personal distribution. Selection of core questions was based on three criteria: applicability (proportion "not applicable"), importance (mean scores on follow-up questions), and comprehensiveness (content coverage, maximum two items per dimension). RESULTS: 1324 questionnaires were returned providing subsample sizes ranging from 52 to 323. Ten questions were excluded because the proportion of "not applicable" responses exceeded 20% in at least one user group. The number of remaining items was reduced to ten by applying the two other criteria. The final short questionnaire included items on outcome (2), clinician services (2), user involvement (2), incorrect treatment (1), information (1), organisation (1), and accessibility (1). CONCLUSION: The Generic Short Patient Experiences Questionnaire (GS-PEQ) is a short, generic set of questions on user experiences with specialist health care that covers important topics for a range of groups. It can be used alone or with other instruments in quality assessment or in research. The psychometric properties and the relevance of the GS-PEQ in other health care settings and countries need further evaluation.