Data extraction from electronic health records (EHRs) for quality measurement of the physical therapy process: comparison between EHR data and survey data.

BACKGROUND: With the emergence of the electronic health records (EHRs) as a pervasive healthcare information technology, new opportunities and challenges for use of clinical data for quality measurements arise with respect to data quality, data availability and comparability. The objective of this study is to test whether data extracted from electronic health records (EHRs) was of comparable quality as survey data for the calculation of quality indicators. METHODS: Data from surveys describing patient cases and filled out by physiotherapists in 2009-2010 were used to calculate scores on eight quality indicators (QIs) to measure the quality of physiotherapy care. In 2011, data was extracted directly from EHRs. The data collection methods were evaluated for comparability. EHR data was compared to survey data on completeness and correctness. RESULTS: Five of the eight QIs could be extracted from the EHRs. Three were omitted from the indicator set, as they proved too difficult to be extracted from the EHRs. Another QI proved incomparable due to errors in the extraction software of some of the EHRs. Three out of four comparable QIs performed better (p < 0.001) in EHR data on completeness. EHR data also proved to be correct; the relative change in indicator scores between EHR and survey data were small (<5 %) in three out of four QIs. CONCLUSION: Data quality of EHRs was sufficient to be used for the calculation of QIs, although comparability to survey data was problematic. Standardization is needed, not only to be able to compare different data collection methods properly, but also to compare between practices with different EHRs. EHRs have the option to administrate narrative data, but natural language processing tools are needed to quantify these text boxes. Such development, can narrow the comparability gap between scoring QIs based on EHR data and based on survey data. EHRs have the potential to provide real time feedback to professionals and quality measurements for research, but more effort is needed to create unambiguous and uniform information and to unlock written text in a standardized manner.

Quality of physical therapy from a patient's perspective; factor analysis on web-based survey data revealed three dimensions on patient experiences with physical therapy.

BACKGROUND: Assessing quality of care from the patient's perspective has changed from patient satisfaction to the more general term patient experience, as satisfaction measures turned out to be less discriminative due to high scores. Literature describes four to ten dimensions of patient experience, tailored to specific conditions or types of care. Given the administrative burden on patients, less dimensions and items could increase feasibility. Ten dimensions of patient experiences with physical therapy (PT) were proposed in the Netherlands in a consensus-based process with patients, physical therapists, health insurers, and policy makers. The aim of this paper is to detect the number of dimensions from data of a field study using factor analysis at item level. METHODS: A web-based survey yielded data of 2,221 patients from 52 PT practices on 41 items. Principal component factor analysis at item level was used to assess the proposed distinction between the ten dimensions. RESULTS: Factor analysis revealed two dimensions: 'personal interaction' and 'practice organisation'. The dimension 'patient reported outcome' was artificially established. The three dimensions 'personal interaction' (14 items) (median(practice level) = 91.1; IQR = 2.4), 'practice organisation' (9 items) (median(practice level) = 88.9; IQR = 6.0) and 'outcome' (3 items) (median(practice level) = 80.6; IQR = 19.5) reduced the number of dimensions from ten to three and the number of items by more than a third. CONCLUSIONS: Factor analysis revealed three dimensions and achieved an item reduction of more than a third. It is a relevant step in the development process of a quality measurement tool to reduce respondent burden, increase clarity, and promote feasibility.

Evaluating quality indicators for physical therapy in primary care.

OBJECTIVE: To evaluate measurement properties of a set of public quality indicators on physical therapy. DESIGN: An observational study with web-based collected survey data (2009 and 2010). SETTING: Dutch primary care physical therapy practices. PARTICIPANTS: In 3743 physical therapy practices, 11 274 physical therapists reporting on 30 patients each. MAIN OUTCOME MEASURES: Eight quality indicators were constructed: screening and diagnostics (n= 2), setting target aim and subsequent of intervention (n = 2), administrating results (n = 1), global outcome measures (n = 2) and patient's treatment agreement (n = 1). Measurement properties on content and construct validity, reproducibility, floor and ceiling effects and interpretability of the indicators were assessed using comparative statistics and multilevel modeling. RESULTS: Content validity was acceptable. Construct validity (using known group techniques) of two outcome indicators was acceptable; hypotheses on age, gender and chronic vs. acute care were confirmed. For the whole set of indicators reproducibility was approximated by correlation of 2009 and 2010 data and rated moderately positive (Spearman's ρ between 0.3 and 0.42 at practice level) and interpretability as acceptable, as distinguishing between patient groups was possible. Ceiling effects were assessed negative as they were high to extremely high (30% for outcome indicator 6-95% for administrating results). CONCLUSION: Weaknesses in data collection should be dealt with to reduce bias and to reduce ceiling effects by randomly extracting data from electronic medical records. More specificity of the indicators seems to be needed, and can be reached by focusing on most prevalent conditions, thus increasing usability of the indicators to improve quality of care.

The Reasons behind the (Non)Use of Feedback Reports for Quality Improvement in Physical Therapy: A Mixed-Method Study.

OBJECTIVES: To explain the use of feedback reports for quality improvements by the reasons to participate in quality measuring projects and to identify barriers and facilitators. DESIGN: Mixed methods design. METHODS: In 2009-2011 a national audit and feedback system for physical therapy (Qualiphy) was initiated in the Netherlands. After each data collection round, an evaluation survey was held amongst its participants. The evaluation survey data was used to explain the use of feedback reports by studying the reasons to participate with Qualiphy with correlation measures and logistic regression. Semi-structured interviews with PTs served to seek confirmation and disentangle barriers and facilitators. RESULTS: Analysis of 257 surveys (response rate: 42.8%) showed that therapists with only financial reasons were less likely to use feedback reports (OR = 0.24;95%CI = 0.11-0.52) compared to therapists with a mixture of reasons. PTs in 2009 and 2010 were more likely to use the feedback reports for quality improvement than PTs in 2011 (OR = 2.41;95%CI = 1.25-4.64 respectively OR = 3.28;95%CI = 1.51-7.10). Changing circumstances in 2011, i.e. using EHRs and financial incentives, had a negative effect on the use of feedback reports (OR = 0.40, 95%CI = 0.20-0.78). Interviews with 12 physical therapists showed that feedback reports could serve as a tool to support and structure quality improvement plans. Barriers were distrust and perceived self-reporting bias on indicator scores. CONCLUSIONS: Implementing financial incentives that are not well-specified and well-targeted can have an adverse effect on using feedback reports to improve quality of care. Distrust is a major barrier to implementing quality systems.