Validity Evidence of the eHealth Literacy Questionnaire (eHLQ) Part 2: Mixed Methods Approach to Evaluate Test Content, Response Process, and Internal Structure in the Australian Community Health Setting.

BACKGROUND: Digital technologies have changed how we manage our health, and eHealth literacy is needed to engage with health technologies. Any eHealth strategy would be ineffective if users' eHealth literacy needs are not addressed. A robust measure of eHealth literacy is essential for understanding these needs. On the basis of the eHealth Literacy Framework, which identified 7 dimensions of eHealth literacy, the eHealth Literacy Questionnaire (eHLQ) was developed. The tool has demonstrated robust psychometric properties in the Danish setting, but validity testing should be an ongoing and accumulative process. OBJECTIVE: This study aims to evaluate validity evidence based on test content, response process, and internal structure of the eHLQ in the Australian community health setting. METHODS: A mixed methods approach was used with cognitive interviewing conducted to examine evidence on test content and response process, whereas a cross-sectional survey was undertaken for evidence on internal structure. Data were collected at 3 diverse community health sites in Victoria, Australia. Psychometric testing included both the classical test theory and item response theory approaches. Methods included Bayesian structural equation modeling for confirmatory factor analysis, internal consistency and test-retest for reliability, and the Bayesian multiple-indicators, multiple-causes model for testing of differential item functioning. RESULTS: Cognitive interviewing identified only 1 confusing term, which was clarified. All items were easy to read and understood as intended. A total of 525 questionnaires were included for psychometric analysis. All scales were homogenous with composite scale reliability ranging from 0.73 to 0.90. The intraclass correlation coefficient for test-retest reliability for the 7 scales ranged from 0.72 to 0.95. A 7-factor Bayesian structural equation modeling using small variance priors for cross-loadings and residual covariances was fitted to the data, and the model of interest produced a satisfactory fit (posterior productive P=.49, 95% CI for the difference between observed and replicated chi-square values -101.40 to 108.83, prior-posterior productive P=.92). All items loaded on the relevant factor, with loadings ranging from 0.36 to 0.94. No significant cross-loading was found. There was no evidence of differential item functioning for administration format, site area, and health setting. However, discriminant validity was not well established for scales 1, 3, 5, 6, and 7. Item response theory analysis found that all items provided precise information at different trait levels, except for 1 item. All items demonstrated different sensitivity to different trait levels and represented a range of difficulty levels. CONCLUSIONS: The evidence suggests that the eHLQ is a tool with robust psychometric properties and further investigation of discriminant validity is recommended. It is ready to be used to identify eHealth literacy strengths and challenges and assist the development of digital health interventions to ensure that people with limited digital access and skills are not left behind.

Validity Evidence Based on Relations to Other Variables of the eHealth Literacy Questionnaire (eHLQ): Bayesian Approach to Test for Known-Groups Validity.

BACKGROUND: As health resources and services are increasingly delivered through digital platforms, eHealth literacy is becoming a set of essential capabilities to improve consumer health in the digital era. To understand eHealth literacy needs, a meaningful measure is required. Strong initial evidence for the reliability and construct validity of inferences drawn from the eHealth Literacy Questionnaire (eHLQ) was obtained during its development in Denmark, but validity testing for varying purposes is an ongoing and cumulative process. OBJECTIVE: This study aims to examine validity evidence based on relations to other variables-using data collected with the known-groups approach-to further explore if the eHLQ is a robust tool to understand eHealth literacy needs in different contexts. A priori hypotheses are set for the expected score differences among age, sex, education, and information and communication technology (ICT) use for each of the 7 eHealth literacy constructs represented by the 7 eHLQ scales. METHODS: A Bayesian mediated multiple indicators multiple causes model approach was used to simultaneously identify group differences and test measurement invariance through differential item functioning across the groups, with ICT use as a mediator. A sample size of 500 participants was estimated. Data were collected at 3 diverse health sites in Australia. RESULTS: Responses from 525 participants were included for analysis. Being older was significantly related to lower scores in 4 eHLQ scales, with 3. Ability to actively engage with digital services having the strongest effect (total effect -0.37; P<.001), followed by 1. Using technology to process health information (total effect -0.32; P<.001), 5. Motivated to engage with digital services (total effect -0.21; P=.01), and 7. Digital services that suit individual needs (total effect -0.21; P=.02). However, the effects were only partially mediated by ICT use. Higher education was associated with higher scores in 1. Using technology to process health information (total effect 0.22; P=.01) and 3. Ability to actively engage with digital services (total effect 0.25; P<.001), with the effects mostly mediated by ICT use. Higher ICT use was related to higher scores in all scales except 2. Understanding health concepts and language and 4. Feel safe and in control. Either no or ignorable cases of differential item functioning were found across the 4 groups. CONCLUSIONS: By using a Bayesian mediated multiple indicators multiple causes model, this study provides supportive validity evidence for the eHLQ based on relations to other variables as well as established evidence regarding internal structure related to measurement invariance across the groups for the 7 scales in the Australian community health context. This study also demonstrates that the eHLQ can be used to gain valuable insights into people's eHealth literacy needs to help optimize access and use of digital health and promote health equity.

Translation method is validity evidence for construct equivalence: analysis of secondary data routinely collected during translations of the Health Literacy Questionnaire (HLQ).

BACKGROUND: Cross-cultural research with patient-reported outcomes measures (PROMs) assumes that the PROM in the target language will measure the same construct in the same way as the PROM in the source language. Yet translation methods are rarely used to qualitatively maximise construct equivalence or to describe the intents of each item to support common understanding within translation teams. This study aimed to systematically investigate the utility of the Translation Integrity Procedure (TIP), in particular the use of item intent descriptions, to maximise construct equivalence during the translation process, and to demonstrate how documented data from the TIP contributes evidence to a validity argument for construct equivalence between translated and source language PROMs. METHODS: Analysis of secondary data was conducted on routinely collected data in TIP Management Grids of translations (n = 9) of the Health Literacy Questionnaire (HLQ) that took place between August 2014 and August 2015: Arabic, Czech, French (Canada), French (France), Hindi, Indonesian, Slovak, Somali and Spanish (Argentina). Two researchers initially independently deductively coded the data to nine common types of translation errors. Round two of coding included an identified 10th code. Coded data were compared for discrepancies, and checked when needed with a third researcher for final code allocation. RESULTS: Across the nine translations, 259 changes were made to provisional forward translations and were coded into 10 types of errors. Most frequently coded errors were Complex word or phrase (n = 99), Semantic (n = 54) and Grammar (n = 27). Errors coded least frequently were Cultural errors (n = 7) and Printed errors (n = 5). CONCLUSIONS: To advance PROM validation practice, this study investigated a documented translation method that includes the careful specification of descriptions of item intents. Assumptions that translated PROMs have construct equivalence between linguistic contexts can be incorrect due to errors in translation. Of particular concern was the use of high level complex words by translators, which, if undetected, could cause flawed interpretation of data from people with low literacy. Item intent descriptions can support translations to maximise construct equivalence, and documented translation data can contribute evidence to justify score interpretation and use of translated PROMS in new linguistic contexts.

Italian validation of the health education impact questionnaire (heiQ) in people with chronic conditions.

BACKGROUND: The health education impact Questionnaire (heiQ) measures eight self-management skills in people with chronic conditions. It seems to be important to provide cross-cultural evidence on its properties in non-English healthcare contexts. The present study assessed the psychometric properties of the heiQ in Italian adults with chronic conditions. METHODS: Two hundred ninety-nine individuals with a chronic condition (mean age = 61.4 years, 50.16% females) completed the heiQ and the Medical Outcomes Study-Short Form (SF-36). Confirmatory factor analyses, Composite Reliability Indices, and bivariate correlations were performed. RESULTS: Structural validity based on 8 correlated factors with good fit was in line with previous research. Concurrent validity was confirmed, as shown by moderate associations between the scores on the Constructive attitudes and approaches, Self-monitoring and insight, Health directed activities, Social integration and support, and Emotional distress subscales and the scores on SF-36 Physical functioning, General health perceptions, Vitality, Social functioning, Perceived mental health and Role limitations due to physical and emotional problems subscales. CONCLUSIONS: The Italian heiQ has strong properties and it can be used routinely also in the Italian healthcare services.

Applying the Electronic Health Literacy Lens: Systematic Review of Electronic Health Interventions Targeted at Socially Disadvantaged Groups.

BACKGROUND: Electronic health (eHealth) has the potential to improve health outcomes. However, eHealth systems need to match the eHealth literacy needs of users to be equitably adopted. Socially disadvantaged groups have lower access and skills to use technologies and are at risk of being digitally marginalized, leading to the potential widening of health disparities. OBJECTIVE: This systematic review aims to explore the role of eHealth literacy and user involvement in developing eHealth interventions targeted at socially disadvantaged groups. METHODS: A systematic search was conducted across 10 databases for eHealth interventions targeted at older adults, ethnic minority groups, low-income groups, low-literacy groups, and rural communities. The eHealth Literacy Framework was used to examine the eHealth literacy components of reviewed interventions. The results were analyzed using narrative synthesis. RESULTS: A total of 51 studies reporting on the results of 48 interventions were evaluated. Most studies were targeted at older adults and ethnic minorities, with only 2 studies focusing on low-literacy groups. eHealth literacy was not considered in the development of any of the studies, and no eHealth literacy assessment was conducted. User involvement in designing interventions was limited, and eHealth intervention developmental frameworks were rarely used. Strategies to assist users in engaging with technical systems were seldom included in the interventions, and accessibility features were limited. The results of the included studies also provided inconclusive evidence on the effectiveness of eHealth interventions. CONCLUSIONS: The findings highlight that eHealth literacy is generally overlooked in developing eHealth interventions targeted at socially disadvantaged groups, whereas evidence about the effectiveness of such interventions is limited. To ensure equal access and inclusiveness in the age of eHealth, eHealth literacy of disadvantaged groups needs to be addressed to help avoid a digital divide. This will assist the realization of recent technological advancements and, importantly, improve health equity.

Development of the Multidimensional Readiness and Enablement Index for Health Technology (READHY) Tool to Measure Individuals' Health Technology Readiness: Initial Testing in a Cancer Rehabilitation Setting.

BACKGROUND: The increasing digitization of health care services with enhanced access to fast internet connections, along with wide use of smartphones, offers the opportunity to get health advice or treatment remotely. For service providers, it is important to consider how consumers can take full advantage of available services and how this can create an enabling environment. However, it is important to consider the digital context and the attributes of current and future users, such as their readiness (ie, knowledge, skills, and attitudes, including trust and motivation). OBJECTIVE: The objective of this study was to evaluate how the eHealth Literacy Questionnaire (eHLQ) combined with selected dimensions from the Health Education Impact Questionnaire (heiQ) and the Health Literacy Questionnaire (HLQ) can be used together as an instrument to characterize an individual's level of health technology readiness and explore how the generated data can be used to create health technology readiness profiles of potential users of health technologies and digital health services. METHODS: We administered the instrument and sociodemographic questions to a population of 305 patients with a recent cancer diagnosis referred to rehabilitation in a setting that plans to introduce various technologies to assist the individuals. We evaluated properties of the Readiness and Enablement Index for Health Technology (READHY) instrument using confirmatory factor analysis, convergent and discriminant validity analysis, and exploratory factor analysis. To identify different health technology readiness profiles in the population, we further analyzed the data using hierarchical and k-means cluster analysis. RESULTS: The confirmatory factor analysis found a suitable fit for the 13 factors with only 1 cross-loading of 1 item between 2 dimensions. The convergent and discriminant validity analysis revealed many factor correlations, suggesting that, in this population, a more parsimonious model might be achieved. Exploratory factor analysis pointed to 5 to 6 constructs based on aggregates of the existing dimensions. The results were not satisfactory, so we performed an 8-factor confirmatory factor analysis, resulting in a good fit with only 1 item cross-loading between 2 dimensions. Cluster analysis showed that data from the READHY instrument can be clustered to create meaningful health technology readiness profiles of users. CONCLUSIONS: The 13 dimensions from heiQ, HLQ, and eHLQ can be used in combination to describe a user's health technology readiness level and degree of enablement. Further studies in other populations are needed to understand whether the associations between dimensions are consistent and the number of dimensions can be reduced.

Application of validity theory and methodology to patient-reported outcome measures (PROMs): building an argument for validity.

BACKGROUND: Data from subjective patient-reported outcome measures (PROMs) are now being used in the health sector to make or support decisions about individuals, groups and populations. Contemporary validity theorists define validity not as a statistical property of the test but as the extent to which empirical evidence supports the interpretation of test scores for an intended use. However, validity testing theory and methodology are rarely evident in the PROM validation literature. Application of this theory and methodology would provide structure for comprehensive validation planning to support improved PROM development and sound arguments for the validity of PROM score interpretation and use in each new context. OBJECTIVE: This paper proposes the application of contemporary validity theory and methodology to PROM validity testing. ILLUSTRATIVE EXAMPLE: The validity testing principles will be applied to a hypothetical case study with a focus on the interpretation and use of scores from a translated PROM that measures health literacy (the Health Literacy Questionnaire or HLQ). DISCUSSION: Although robust psychometric properties of a PROM are a pre-condition to its use, a PROM's validity lies in the sound argument that a network of empirical evidence supports the intended interpretation and use of PROM scores for decision making in a particular context. The health sector is yet to apply contemporary theory and methodology to PROM development and validation. The theoretical and methodological processes in this paper are offered as an advancement of the theory and practice of PROM validity testing in the health sector.

A Multidimensional Tool Based on the eHealth Literacy Framework: Development and Initial Validity Testing of the eHealth Literacy Questionnaire (eHLQ).

BACKGROUND: For people to be able to access, understand, and benefit from the increasing digitalization of health services, it is critical that services are provided in a way that meets the user's needs, resources, and competence. OBJECTIVE: The objective of the study was to develop a questionnaire that captures the 7-dimensional eHealth Literacy Framework (eHLF). METHODS: Draft items were created in parallel in English and Danish. The items were generated from 450 statements collected during the conceptual development of eHLF. In all, 57 items (7 to 9 items per scale) were generated and adjusted after cognitive testing. Items were tested in 475 people recruited from settings in which the scale was intended to be used (community and health care settings) and including people with a range of chronic conditions. Measurement properties were assessed using approaches from item response theory (IRT) and classical test theory (CTT) such as confirmatory factor analysis (CFA) and reliability using composite scale reliability (CSR); potential bias due to age and sex was evaluated using differential item functioning (DIF). RESULTS: CFA confirmed the presence of the 7 a priori dimensions of eHLF. Following item analysis, a 35-item 7-scale questionnaire was constructed, covering (1) using technology to process health information (5 items, CSR=.84), (2) understanding of health concepts and language (5 items, CSR=.75), (3) ability to actively engage with digital services (5 items, CSR=.86), (4) feel safe and in control (5 items, CSR=.87), (5) motivated to engage with digital services (5 items, CSR=.84), (6) access to digital services that work (6 items, CSR=.77), and (7) digital services that suit individual needs (4 items, CSR=.85). A 7-factor CFA model, using small-variance priors for cross-loadings and residual correlations, had a satisfactory fit (posterior productive P value: .27, 95% CI for the difference between the observed and replicated chi-square values: -63.7 to 133.8). The CFA showed that all items loaded strongly on their respective factors. The IRT analysis showed that no items were found to have disordered thresholds. For most scales, discriminant validity was acceptable; however, 2 pairs of dimensions were highly correlated; dimensions 1 and 5 (r=.95), and dimensions 6 and 7 (r=.96). All dimensions were retained because of strong content differentiation and potential causal relationships between these dimensions. There is no evidence of DIF. CONCLUSIONS: The eHealth Literacy Questionnaire (eHLQ) is a multidimensional tool based on a well-defined a priori eHLF framework with robust properties. It has satisfactory evidence of construct validity and reliable measurement across a broad range of concepts (using both CTT and IRT traditions) in various groups. It is designed to be used to understand and evaluate people's interaction with digital health services.

Validation of the dutch version of the health education impact questionnaire (HEIQ) and comparison of the Dutch translation with the English, German and French HEIQ.

BACKGROUND: The Health Education Impact Questionnaire (heiQ) evaluates the effectiveness of health education and self-management programs provided to people dealing with a wide range of conditions. Aim of this study was to translate, culturally adapt and validate the Dutch translation of the heiQ and to compare the results with the English, German and French translations. METHODS: A systematic translation process was undertaken. Psychometric properties were studied among patients with arthritis, atopic dermatitis, food allergy and asthma (n = 286). Factorial validity using confirmatory factor analysis, item difficulty (D), item remainder correlation and composite reliability were conducted. Stability was tested using the intra-class correlation coefficient (ICC). RESULTS: Items were well understood and only minor language adjustments were required. Confirmatory fit indices were >0.95 and item difficulty was D ≥ 0.65 for all items in scales showing acceptable fit indices, except for the reversed Emotional distress scale. Composite reliability ranged between 0.67 and 0.85. Test-retest reliability (n = 93) ICC varied between 0.61 and 0.84. Comparisons with other translations showed comparable fit indices. A lower ICC on Self-monitoring and insight scale was observed. CONCLUSIONS: The Dutch translation of the heiQ was found to be well understood and user friendly by patients with Rheumatoid Arthritis, Atopic Dermatitis, Food allergy and asthma and to have robust psychometric properties for evaluating the impact of health education and self-management programs. Given the wide applications of the heiQ and the comparability of the Dutch results with the English, German and French version, the heiQ is a practical and useful questionnaire to evaluate the impact of self-management support programs in different countries and populations with different diseases.

Systematic development and implementation of interventions to OPtimise Health Literacy and Access (Ophelia).

BACKGROUND: The need for healthcare strengthening to enhance equity is critical, requiring systematic approaches that focus on those experiencing lesser access and outcomes. This project developed and tested the Ophelia (OPtimising HEalth LIteracy and Access) approach for co-design of interventions to improve health literacy and equity of access. Eight principles guided this development: Outcomes focused; Equity driven, Needs diagnosis, Co-design, Driven by local wisdom, Sustainable, Responsive and Systematically applied. We report the application of the Ophelia process where proof-of-concept was defined as successful application of the principles. METHODS: Nine sites were briefed on the aims of the project around health literacy, co-design and quality improvement. The sites were rural/metropolitan, small/large hospitals, community health centres or municipalities. Each site identified their own priorities for improvement; collected health literacy data using the Health Literacy Questionnaire (HLQ) within the identified priority groups; engaged staff in co-design workshops to generate ideas for improvement; developed program-logic models; and implemented their projects using Plan-Do-Study-Act (PDSA) cycles. Evaluation included assessment of impacts on organisations, practitioners and service users, and whether the principles were applied. RESULTS: Sites undertook co-design workshops involving discussion of service user needs informed by HLQ (n = 813) and interview data. Sites generated between 21 and 78 intervention ideas and then planned their selected interventions through program-logic models. Sites successfully implemented interventions and refined them progressively with PDSA cycles. Interventions generally involved one of four pathways: development of clinician skills and resources for health literacy, engagement of community volunteers to disseminate health promotion messages, direct impact on consumers' health literacy, and redesign of existing services. Evidence of application of the principles was found in all sites. CONCLUSIONS: The Ophelia approach guided identification of health literacy issues at each participating site and the development and implementation of locally appropriate solutions. The eight principles provided a framework that allowed flexible application of the Ophelia approach and generation of a diverse set of interventions. Changes were observed at organisational, staff, and community member levels. The Ophelia approach can be used to generate health service improvements that enhance health outcomes and address inequity of access to healthcare.

The Health Literacy Questionnaire (HLQ) at the patient-clinician interface: a qualitative study of what patients and clinicians mean by their HLQ scores.

BACKGROUND: The Health Literacy Questionnaire (HLQ) has nine scales that each measure an aspect of the multidimensional construct of health literacy. All scales have good psychometric properties. However, it is the interpretations of data within contexts that must be proven valid, not just the psychometric properties of a measurement instrument. The purpose of this study was to establish the extent of concordance and discordance between individual patient and clinician interpretations of HLQ data in the context of complex case management. METHODS: Sixteen patients with complex needs completed the HLQ and were interviewed to discuss the reasons for their answers. Also, the clinicians of each of these patients completed the HLQ about their patient, and were interviewed to discuss the reasons for their answers. Thematic analysis of HLQ scores and interview data determined the extent of concordance between patient and clinician HLQ responses, and the reasons for discordance. RESULTS: Highest concordance (80%) between patient and clinician item-response pairs was seen in Scale 1 and highest discordance (56%) was seen in Scale 6. Four themes were identified to explain discordance: 1) Technical or literal meaning of specific words; 2) Patients' changing or evolving circumstances; 3) Different expectations and criteria for assigning HLQ scores; and 4) Different perspectives about a patient's reliance on healthcare providers. CONCLUSION: This study shows that the HLQ can act as an adjunct to clinical practice to help clinicians understand a patient's health literacy challenges and strengths early in a clinical encounter. Importantly, clinicians can use the HLQ to detect differences between their own perspectives about a patient's health literacy and the patient's perspective, and to initiate discussion to explore this. Provision of training to better detect these differences may assist clinicians to provide improved care. The outcomes of this study contribute to the growing body of international validation evidence about the use of the HLQ in different contexts. More specifically, this study has shown that the HLQ has measurement veracity at the patient and clinician level and may support clinicians to understand patients' health literacy and enable a deeper engagement with healthcare services.

Measuring health literacy in community agencies: a Bayesian study of the factor structure and measurement invariance of the health literacy questionnaire (HLQ).

BACKGROUND: The development of the Health Literacy Questionnaire (HLQ), reported in 2013, attracted widespread international interest. While the original study samples were drawn from clinical and home-based aged-care settings, the HLQ was designed for the full range of healthcare contexts including community-based health promotion and support services. We report a follow-up study of the psychometric properties of the HLQ with respondents from a diverse range of community-based organisations with the principal goal of contributing to the development of a soundly validated evidence base for its use in community health settings. METHODS: Data were provided by 813 clients of 8 community agencies in Victoria, Australia who were administered the HLQ during the needs assessment stage of the Ophelia project, a health literacy-based intervention. Most analyses were conducted using Bayesian structural equation modelling that enables rigorous analysis of data but with some relaxation of the restrictive requirements for zero cross-loadings and residual correlations of 'classical' confirmatory factor analysis. Scale homogeneity was investigated with one-factor models that allowed for the presence of small item residual correlations while discriminant validity was studied using the inter-factor correlations and factor loadings from a full 9-factor model with similar allowance for small residual correlations and cross-loadings. Measurement invariance was investigated scale-by-scale using a model that required strict invariance of item factor loadings, thresholds, residual variances and co-variances. RESULTS: All HLQ scales were found to be homogenous with composite reliability ranging from 0.80 to 0.89. The factor structure of the HLQ was replicated and 6 of the 9 scales were found to exhibit clear-cut discriminant validity. With a small number of exceptions involving non-invariance of factor loadings, strict measurement invariance was established across the participating organisations and the gender, language background, age and educational level of respondents. CONCLUSIONS: The HLQ is highly reliable, even with only 4 to 6 items per scale. It provides unbiased mean estimates of group differences across key demographic indicators. While measuring relatively narrow constructs, the 9 dimensions are clearly separate and therefore provide fine-grained data on the multidimensional area of health literacy. These analyses provide researchers, program managers and policymakers with a range of robust evidence by which they can make judgements about the appropriate use of the HLQ for their community-based setting.

Distribution of health literacy strengths and weaknesses across socio-demographic groups: a cross-sectional survey using the Health Literacy Questionnaire (HLQ).

BACKGROUND: Recent advances in the measurement of health literacy allow description of a broad range of personal and social dimensions of the concept. Identifying differences in patterns of health literacy between population sub-groups will increase understanding of how health literacy contributes to health inequities and inform intervention development. The aim of this study was to use a multi-dimensional measurement tool to describe the health literacy of adults in urban and rural Victoria, Australia. METHODS: Data were collected from clients (n = 813) of 8 health and community care organisations, using the Health Literacy Questionnaire (HLQ). Demographic and health service data were also collected. Data were analysed using descriptive statistics. Effect sizes (ES) for standardised differences in means were used to describe the magnitude of difference between demographic sub-groups. RESULTS: Mean age of respondents was 72.1 (range 19-99) years. Females comprised 63% of the sample, 48% had not completed secondary education, and 96% reported at least one existing health condition. Small to large ES were seen for mean differences in HLQ scales between most demographic groups. Compared with participants who spoke English at home, those not speaking English at home had much lower scores for most HLQ scales including the scales 'Understanding health information well enough to know what to do' (ES -1.09 [95% confidence interval (CI) -1.33 to -0.84]), 'Ability to actively engage with healthcare providers' (ES -1.00 [95% CI -1.24, -0.75]), and 'Navigating the healthcare system' (ES -0.72 [95% CI -0.97, -0.48]). Similar patterns and ES were seen for participants born overseas compared with those born in Australia. Smaller ES were seen for sex, age group, private health insurance status, number of chronic conditions, and living alone. CONCLUSIONS: This study has revealed some large health literacy differences across nine domains of health literacy in adults using health services in Victoria. These findings provide insights into the relationship between health literacy and socioeconomic position in vulnerable groups and, given the focus of the HLQ, provide guidance for the development of equitable interventions.

The OPtimising HEalth LIterAcy (Ophelia) process: study protocol for using health literacy profiling and community engagement to create and implement health reform.

BACKGROUND: Health literacy is a multi-dimensional concept comprising a range of cognitive, affective, social, and personal skills and attributes. This paper describes the research and development protocol for a large communities-based collaborative project in Victoria, Australia that aims to identify and respond to health literacy issues for people with chronic conditions. The project, called Ophelia (OPtimising HEalth LIterAcy) Victoria, is a partnership between two universities, eight service organisations and the Victorian Government. Based on the identified issues, it will develop and pilot health literacy interventions across eight disparate health services to inform the creation of a health literacy response framework to improve health outcomes and reduce health inequalities. METHODS/DESIGN: The protocol draws on many inputs including the experience of the partners in previous co-creation and roll-out of large-scale health-promotion initiatives. Three key conceptual models/discourses inform the protocol: intervention mapping; quality improvement collaboratives, and realist synthesis. The protocol is outcomes-oriented and focuses on two key questions: 'What are the health literacy strengths and weaknesses of clients of participating sites?', and 'How do sites interpret and respond to these in order to achieve positive health and equity outcomes for their clients?'. The process has six steps in three main phases. The first phase is a needs assessment that uses the Health Literacy Questionnaire (HLQ), a multi-dimensional measure of health literacy, to identify common health literacy needs among clients. The second phase involves front-line staff and management within each service organisation in co-creating intervention plans to strategically respond to the identified local needs. The third phase will trial the interventions within each site to determine if the site can improve identified limitations to service access and/or health outcomes. DISCUSSION: There have been few attempts to assist agencies to identify, and respond, in a planned way, to the varied health literacy needs of their clients. This project will assess the potential for targeted, locally-developed health literacy interventions to improve access, equity and outcomes.

Absence of social desirability bias in the evaluation of chronic disease self-management interventions.

BACKGROUND: Bias due to social desirability has long been of concern to evaluators relying on self-report data. It is conceivable that health program evaluation is particularly susceptible to social desirability bias as individuals may be inclined to present themselves or certain health behaviors in a more positive light and/or appease the course leader. Thus, the influence of social desirability bias on self-report outcomes was explored in the present study. METHODS: Data were collected from 331 participants of group-based chronic disease self-management interventions using the highly robust eight-scale Health Education Impact Questionnaire (heiQ) and the 13-item short form Marlowe-Crowne Social Desirability Scale (MC-C). The majority of self-management courses were run by community-based organizations across Australia between February 2005 and December 2006 where 6 to 12 individuals have the opportunity to develop considerable rapport with course leaders and each other over about six weeks. Pre-test data were collected on the first day of courses, while post-test and social desirability scores were assessed at the end of courses. A model of partial mediation within the framework of structural equation modeling was developed with social desirability as the mediating variable between pre-test and post-test. RESULTS: The 'Defensiveness' factor of the MC-C showed clear association with heiQ pre-test data, a prerequisite for investigating mediation; however, when investigating the eight full pre-test/post-test models 'Defensiveness' was only associated with one heiQ scale. This effect was small, explaining 8% of the variance in the model. No other meditational effects through social desirability were observed. CONCLUSIONS: The overall lack of association of social desirability with heiQ outcomes was surprising as it had been expected that it would explain at least some of the variance observed between pre-test and post-test. With the assumption that the MC-C captures the propensity for an individual to provide socially desirable answers, this study concludes that change scores in chronic disease self-management program evaluation are not biased by social desirability.

Measurement issues in the evaluation of chronic disease self-management programs.

OBJECTIVES: To provide an in-depth analysis of outcome measures used in the evaluation of chronic disease self-management programs consistent with the Stanford curricula. METHODS: Based on a systematic review on self-management programs, effect sizes derived from reported outcome measures are categorized according to the quality of life appraisal model developed by Schwartz and Rapkin which classifies outcomes from performance-based measures (e.g., clinical outcomes) to evaluation-based measures (e.g., emotional well-being). RESULTS: The majority of outcomes assessed in self-management trials are based on evaluation-based methods. Overall, effects on knowledge--the only performance-based measure observed in selected trials--are generally medium to large. In contrast, substantially more inconsistent results are found for both perception- and evaluation-based measures that mostly range between nil and small positive effects. CONCLUSIONS: Effectiveness of self-management interventions and resulting recommendations for health policy makers are most frequently derived from highly variable evaluation-based measures, that is, types of outcomes that potentially carry a substantial amount of measurement error and/or bias such as response shift. Therefore, decisions regarding the value and efficacy of chronic disease self-management programs need to be interpreted with care. More research, especially qualitative studies, is needed to unravel cognitive processes and the role of response shift bias in the measurement of change.

The grounded psychometric development and initial validation of the Health Literacy Questionnaire (HLQ).

BACKGROUND: Health literacy has become an increasingly important concept in public health. We sought to develop a comprehensive measure of health literacy capable of diagnosing health literacy needs across individuals and organisations by utilizing perspectives from the general population, patients, practitioners and policymakers. METHODS: Using a validity-driven approach we undertook grounded consultations (workshops and interviews) to identify broad conceptually distinct domains. Questionnaire items were developed directly from the consultation data following a strict process aiming to capture the full range of experiences of people currently engaged in healthcare through to people in the general population. Psychometric analyses included confirmatory factor analysis (CFA) and item response theory. Cognitive interviews were used to ensure questions were understood as intended. Items were initially tested in a calibration sample from community health, home care and hospital settings (N=634) and then in a replication sample (N=405) comprising recent emergency department attendees. RESULTS: Initially 91 items were generated across 6 scales with agree/disagree response options and 5 scales with difficulty in undertaking tasks response options. Cognitive testing revealed that most items were well understood and only some minor re-wording was required. Psychometric testing of the calibration sample identified 34 poorly performing or conceptually redundant items and they were removed resulting in 10 scales. These were then tested in a replication sample and refined to yield 9 final scales comprising 44 items. A 9-factor CFA model was fitted to these items with no cross-loadings or correlated residuals allowed. Given the very restricted nature of the model, the fit was quite satisfactory: χ²WLSMV(866 d.f.) = 2927, p<0.000, CFI = 0.936, TLI = 0.930, RMSEA = 0.076, and WRMR = 1.698. Final scales included: Feeling understood and supported by healthcare providers; Having sufficient information to manage my health; Actively managing my health; Social support for health; Appraisal of health information; Ability to actively engage with healthcare providers; Navigating the healthcare system; Ability to find good health information; and Understand health information well enough to know what to do. CONCLUSIONS: The HLQ covers 9 conceptually distinct areas of health literacy to assess the needs and challenges of a wide range of people and organisations. Given the validity-driven approach, the HLQ is likely to be useful in surveys, intervention evaluation, and studies of the needs and capabilities of individuals.

Construction of the descriptive system for the Assessment of Quality of Life AQoL-6D utility instrument.

BACKGROUND: Multi attribute utility (MAU) instruments are used to include the health related quality of life (HRQoL) in economic evaluations of health programs. Comparative studies suggest different MAU instruments measure related but different constructs. The objective of this paper is to describe the methods employed to achieve content validity in the descriptive system of the Assessment of Quality of Life (AQoL)-6D, MAU instrument. METHODS: The AQoL program introduced the use of psychometric methods in the construction of health related MAU instruments. To develop the AQoL-6D we selected 112 items from previous research, focus groups and expert judgment and administered them to 316 members of the public and 302 hospital patients. The search for content validity across a broad spectrum of health states required both formative and reflective modelling. We employed Exploratory Factor Analysis and Structural Equation Modelling (SEM) to meet these dual requirements. RESULTS AND DISCUSSION: The resulting instrument employs 20 items in a multi-tier descriptive system. Latent dimension variables achieve sensitive descriptions of 6 dimensions which, in turn, combine to form a single latent QoL variable. Diagnostic statistics from the SEM analysis are exceptionally good and confirm the hypothesised structure of the model. CONCLUSIONS: The AQoL-6D descriptive system has good psychometric properties. They imply that the instrument has achieved construct validity and provides a sensitive description of HRQoL. This means that it may be used with confidence for measuring health related quality of life and that it is a suitable basis for modelling utilities for inclusion in the economic evaluation of health programs.

The inclusion of 'then-test' questions in post-test questionnaires alters post-test responses: a randomized study of bias in health program evaluation.

OBJECTIVES: Program evaluations are frequently based on 'then-test' data, i.e., pre-test collected in retrospect. While the application of the then-test has practical advantages, little is known about the validity of then-test data. Because of the collection of then-test in close proximity to post-test questions, this study was aimed at exploring whether the presence of then-test questions in post-test questionnaires influenced subjects' responses to post-test. PATIENTS AND METHODS: To test the influence of then-test questions, we designed a randomized three-group study in the context of chronic disease self-management programs. Interventions had comparable goals and philosophies, and all 949 study participants filled out identical Health Education Impact Questionnaires (heiQ) at pre-test. At post-test, participants were then randomized to one of the following three groups: Group A responded to post-test questions only (n = 331); Group B filled out transition questions in addition to post-test (n = 304); and Group C filled out then-test questions in addition to post-test (n = 314). RESULTS: Significant post-test differences were found in six of eight heiQ scales, with respondents who filled out then-test questions reporting significantly higher post-test scores than respondents of the other groups. CONCLUSIONS: This study provides evidence that the inclusion of then-test questions alters post-test responses, suggesting that change scores based on then-test data be interpreted with care.

eHealth Literacy of Australian Undergraduate Health Profession Students: A Descriptive Study.

Rapid growth in digital health technologies has increased demand for eHealth literacy of all stakeholders within health and social care environments. The digital future of health care services requires the next generation of health professionals to be well-prepared to confidently provide high-quality and safe health care. The aim of this study was to explore the eHealth literacy of undergraduate health profession students to inform undergraduate curriculum development to promote work-readiness. A cross-sectional survey was undertaken at an Australian university using the seven-domain eHealth Literacy Questionnaire (eHLQ), with 610 students participating. A one-way Multivariate Analysis of Variance (MANOVA) with follow-up univariate analysis (ANOVA) was used to determine if there were differences in eHLQ scores across 11 sociodemographic variables. Students generally had good knowledge of health (Scale 2); however, they had concerns over the security of online health data (Scale 4). There were also significant differences in age and ownership of digital devices. Students who were younger reported higher scores across all seven eHLQ scales than older students. This research provided an understanding of eHealth literacy of health profession students and revealed sub-groups that have lower eHealth literacy, suggesting that digital health skills should be integrated into university curriculums, especially related to practice-based digital applications with special focus to address privacy and security concerns. Preparation of health profession students so they can efficiently address their own needs, and the needs of others, is recommended to minimise the digital divide within health and social care environments.