Developing a questionnaire to evaluate an automated audit & feedback intervention: a Rand-modified Delphi method.

BACKGROUND: Audit and feedback (A&F) is a widely used implementation strategy to evaluate and improve medical practice. The optimal design of an A&F system is uncertain and structured process evaluations are currently lacking. This study aimed to develop and validate a questionnaire to evaluate the use of automated A&F systems. METHODS: Based on the Clinical Performance Feedback Intervention Theory (CP-FIT) and the REFLECT-52 (REassessing audit & Feedback interventions: a tooL for Evaluating Compliance with suggested besT practices) evaluation tool a questionnaire was designed for the purpose of evaluating automated A&F systems. A Rand-modified Delphi method was used to develop the process evaluation and obtain validation. Fourteen experts from different domains in primary care consented to participate and individually scored the questions on a 9-point Likert scale. Afterwards, the questions were discussed in a consensus meeting. After approval, the final questionnaire was compiled. RESULTS: A 34-question questionnaire composed of 57 items was developed and presented to the expert panel. The consensus meeting resulted in a selection of 31 questions, subdivided into 43 items. A final list of 30 questions consisting of 42 items was obtained. CONCLUSION: A questionnaire consisting of 30 questions was drawn up for the assessment and improvement of automated A&F systems, based on CP-FIT and REFLECT-52 theory and approved by experts. Next steps will be piloting and implementation of the questionnaire.

The Impact of the COVID-19 Pandemic on the Registration and Care Provision of Mental Health Problems in General Practice: Registry-Based Study.

BACKGROUND: The impact of the COVID-19 pandemic on mental health in general practice remains uncertain. Several studies showed an increase in terms of mental health problems during the pandemic. In Belgium, especially during the first waves of the pandemic, access to general practice was limited. Specifically, it is unclear how this impacted not only the registration of mental health problems itself but also the care for patients with an existing mental health problem. OBJECTIVE: This study aimed to know the impact of the COVID-19 pandemic on (1) the incidence of newly registered mental health problems and (2) the provision of care for patients with mental health problems in general practice, both using a pre-COVID-19 baseline. METHODS: The prepandemic volume of provided care (care provision) for patients with mental health problems was compared to that from 2020-2021 by using INTEGO, a Belgian general practice morbidity registry. Care provision was defined as the total number of new registrations in a patient's electronic medical record. Regression models evaluated the association of demographic factors and care provision in patients with mental health problems, both before and during the pandemic. RESULTS: During the COVID-19 pandemic as compared to before the COVID-19 pandemic, the incidence of registered mental health problems showed a fluctuating course, with a sharp drop in registrations during the first wave. Registrations for depression and anxiety increased, whereas the incidence of registered eating disorders, substance abuse, and personality problems decreased. During the 5 COVID-19 waves, the overall incidence of registered mental health problems dropped during the wave and rose again when measures were relaxed. A relative rise of 8.7% and 40% in volume of provided care, specifically for patients with mental health problems, was seen during the first and second years of the COVID-19 pandemic, respectively. Care provision for patients with mental health problems was higher in older patients, male patients, patients living in center cities (centrumsteden), patients with lower socioeconomic status (SES), native Belgian patients, and patients with acute rather than chronic mental health problems. Compared to prepandemic care provision, a reduction of 10% was observed in people with a low SES. CONCLUSIONS: This study showed (1) a relative overall increase in the registrations of mental health problems in general practice and (2) an increase in care provision for patients with mental health problems in the first 2 years of the COVID-19 pandemic. Low SES remained a determining factor for more care provision, but care provision dropped significantly in people with mental health problems with a low SES. Our findings suggest that the pandemic in Belgium was also largely a "syndemic," affecting different layers of the population disproportionately.

The impact of the Covid-19 pandemic on the incidence of diseases and the provision of primary care: A registry-based study.

INTRODUCTION: The Covid-19 pandemic had a tremendous impact on healthcare but uncertainty remains about the extent to which primary care provision was affected. Therefore, this paper aims to assess the impact on primary care provision and the evolution of the incidence of disease during the first year of the Covid-19 pandemic in Flanders (Belgium). METHODS: Care provision was defined as the number of new entries added to a patient's medical history. Pre-pandemic care provision (February 1, 2018-January 31, 2020) was compared with care provision during the pandemic (February 1, 2020-January 31, 2021). A large morbidity registry (Intego) was used. Regression models compared the effect of demographic characteristics on care provision and on acute and chronic diagnoses incidence both prior and during the pandemic. RESULTS: During the first year of the Covid-19 pandemic, overall care provision increased with 9.1% (95%CI 8.5%;9.6%). There was an increase in acute diagnoses of 5.1% (95%CI 4.2%;6.0%) and a decrease in the selected chronic diagnoses of 12.8% (95% CI 7.0%;18.4%). Obesity was an exception with an overall incidence increase. The pandemic led to strong fluctuations in care provision that were not the same for all types of care and all demographic groups in Flanders. Relative to other groups in the population, the pandemic caused a reduction in care provision for children aged 0-17 year and patients from a lower socio-economic situation. CONCLUSION: This paper strengthened the claim that Covid-19 should be considered as a syndemic instead of a pandemic. During the first Covid-19 year, overall care provision and the incidence of acute diagnoses increased, whereas chronic diseases' incidence decreased, except for obesity diagnoses which increased. More granular, care provision and chronic diseases' incidence decreased during the lockdowns, especially for people with a lower socio-economic status. After the lockdowns they both returned to baseline.

Development of primary care quality indicators for chronic obstructive pulmonary disease using a Delphi-derived method.

High-quality care for patients with COPD is necessary. To achieve quality improvement in primary care, the general practitioner and the electronic health record (EHR) play an important role. The aim of this study was to develop a set of evidence-based and EHR extractable quality indicators (QIs) to measure and improve the quality of COPD primary care. We composed a multidisciplinary expert panel of 12 members, including patients, and used a RAND-modified Delphi method. The SMART principle was applied to select recommendations and QIs from international guidelines as well as existing sets of QIs, and these recommendations and QIs were added to an individual written questionnaire. Based on the median score, prioritization and degree of agreement, the recommendations and QIs were rated as having a high, uncertain or low potential to measure the quality of COPD primary care and were then discussed in an online consensus meeting for inclusion or exclusion. After a final validation, a core set of recommendations was translated into QIs. From 37 recommendations, obtained out of 10 international guidelines, and 5 existing indicators, a core set of 18 recommendations and 2 QIs was derived after the rating procedure. The expert panel added one new recommendation. Together, the recommendations and QIs were translated and merged into a final set of 21 QIs. Our study developed a set of 21 evidence-based and EHR-extractable QIs for COPD in primary care. These indicators can be used in an automated quality assessment to measure and improve the quality of COPD primary care.

The effect of automated audit and feedback on data completeness in the electronic health record of the general physician: protocol for a cluster randomized controlled trial.

BACKGROUND: The electronic health record (EHR) of the general physician (GP) is an important tool that can be used to assess and improve the quality of healthcare. However, there are some problems when (re) using the data gathered in the EHR for quality assessments. One problem is the lack of data completeness in the EHR. Audit and feedback (A&F) is a well-known quality intervention that can improve the quality of healthcare. We hypothesize that an automated A&F intervention can be adapted to improve the data completeness of the EHR of the GP, more specifically, the number of correctly registered diagnoses of type 2 diabetes and chronic kidney disease. METHODS: This study is a pragmatic cluster randomized controlled trial with an intervention at the level of GP practice. The intervention consists of an audit and extended electronically delivered feedback with multiple components that will be delivered 4 times electronically to general practices over 12 months. The data will be analyzed on an aggregated level (per GP practice). The primary outcome is the percentage of correctly registered diagnoses of type 2 diabetes. The key secondary outcome is the registration of chronic kidney disease. Exploratory secondary outcomes are the registration of heart failure, biometric data and lifestyle habits, and the evolution of 4 different EHR-extractable quality indicators. DISCUSSION: This cluster randomized controlled trial intends to primarily improve the registration of type 2 diabetes in the EHR of the GP and to secondarily improve the registration of chronic kidney disease. In addition, the registration of heart failure, lifestyle parameters, and biometric data in the EHR of the GP are explored together with 4 EHR-extractable quality indicators. By doing so, this study aims to improve the data completeness of the EHR, paving the way for future quality assessments. TRIAL REGISTRATION: ClinicalTrials.gov NCT04388228 . Registered on May 14, 2020.

The effect of electronic audits and feedback in primary care and factors that contribute to their effectiveness: a systematic review.

PURPOSE: The aim of this systematic review was (i) to assess whether electronic audit and feedback (A&F) is effective in primary care and (ii) to evaluate important features concerning content and delivery of the feedback in primary care, including the use of benchmarks, the frequency of feedback, the cognitive load of feedback and the evidence-based aspects of the feedback. DATA SOURCES: The MEDLINE, Embase, CINAHL and CENTRAL databases were searched for articles published since 2010 by replicating the search strategy used in the last Cochrane review on A&F. STUDY SELECTION: Two independent reviewers assessed the records for their eligibility, performed the data extraction and evaluated the risk of bias. Our search resulted in 8744 records, including the 140 randomized controlled trials (RCTs) from the last Cochrane Review. The full texts of 431 articles were assessed to determine their eligibility. Finally, 29 articles were included. DATA EXTRACTION: Two independent reviewers extracted standard data, data on the effectiveness and outcomes of the interventions, data on the kind of electronic feedback (static versus interactive) and data on the aforementioned feedback features. RESULTS OF DATA SYNTHESIS: Twenty-two studies (76%) showed that electronic A&F was effective. All interventions targeting medication safety, preventive medicine, cholesterol management and depression showed an effect. Approximately 70% of the included studies used benchmarks and high-quality evidence in the content of the feedback. In almost half of the studies, the cognitive load of feedback was not reported. Due to high heterogeneity in the results, no meta-analysis was performed. CONCLUSION: This systematic review included 29 articles examining electronic A&F interventions in primary care, and 76% of the interventions were effective. Our findings suggest electronic A&F is effective in primary care for different conditions such as medication safety and preventive medicine. Some of the benefits of electronic A&F include its scalability and the potential to be cost effective. The use of benchmarks as comparators and feedback based on high-quality evidence are widely used and important features of electronic feedback in primary care. However, other important features such as the cognitive load of feedback and the frequency of feedback provision are poorly described in the design of many electronic A&F intervention, indicating that a better description or implementation of these features is needed. Developing a framework or methodology for automated A&F interventions in primary care could be useful for future research.

Developing quality indicators for Chronic Kidney Disease in primary care, extractable from the Electronic Medical Record. A Rand-modified Delphi method.

BACKGROUND: Chronic kidney disease (CKD) is a common chronic condition and a rising public health issue with increased morbidity and mortality, even at an early stage. Primary care has a pivotal role in the early detection and in the integrated management of CKD which should be of high quality. The quality of care for CKD can be assessed using quality indicators (QIs) and if these QIs are extractable from the electronic medical record (EMR) of the general physician, the number of patients whose quality of care can be evaluated, could increase vastly. Therefore the aim of this study is to develop QIs which are evidence based, EMR extractable and which can be used as a framework to automate quality assessment. METHODS: We used a Rand-modified Delphi method to develop QIs for CKD in primary care. A questionnaire was designed by extracting recommendations from international guidelines based on the SMART principle and the EMR extractability. A multidisciplinary expert panel, including patients, individually scored the recommendations for measuring high quality care on a 9-point Likert scale. The results were analyzed based on the median Likert score, prioritization and agreement. Subsequently, the recommendations were discussed in a consensus meeting for their in- or exclusion. After a final appraisal by the panel members this resulted in a core set of recommendations, which were then transformed into QIs. RESULTS: A questionnaire composed of 99 recommendations was extracted from 10 international guidelines. The consensus meeting resulted in a core set of 36 recommendations that were translated into 36 QIs. This final set consists of QIs concerning definition & classification, screening, diagnosis, management consisting of follow up, treatment & vaccination, medication & patient safety and referral to a specialist. It were mostly the patients participating in the panel who stressed the importance of the QIs concerning medication & patient safety and a timely referral to a specialist. CONCLUSION: This study provides a set of 36 EMR extractable QIs for measuring the quality of primary care for CKD. These QIs can be used as a framework to automate quality assessment for CKD in primary care.

Development of quality indicators for type 2 diabetes, extractable from the electronic health record of the general physician. A rand-modified Delphi method.

AIMS: Type 2 diabetes mellitus is a worldwide cause of significant morbidity and mortality. The general physician (GP) plays an important role in managing this disease and the use of the electronic health record (EHR) can improve quality for type 2 diabetes care. We aimed to develop a set of evidence-based and EHR extractable quality indicators for type 2 diabetes, enabling an automated quality assessment. METHODS: We used a Rand-modified Delphi method. Recommendations were selected from (inter)national guidelines using the 'SMART' principle and scored by a multidisciplinary expert panel. After analysis of the median score, prioritization and consensus, recommendations were discussed during a consensus meeting. A final validation round resulted in a core set of recommendations, which were transformed into quality indicators. RESULTS: A total of 101 recommendations originating from 10 (inter)national guidelines were presented to the expert panel, which resulted in a core set of 50 recommendations that were merged and modified into 36 recommendations after the consensus meeting. The panel added 3 recommendations. This resulted in a final set of 39 quality indicators. CONCLUSIONS: Our study presents a set of 39 quality indicators for type 2 diabetes in primary care that are EHR extractable, enabling automated quality assessment.

Defining quality indicators for heart failure in general practice.

Background: Quality indicators (QIs) are used to measure and evaluate quality of care. However, QIs to evaluate care for HF patients in general practice in Belgium are lacking. Therefore, this study aimed to determine which QIs, rooted in the electronic health record (EHR), are useful to monitor quality of care for patients with HF in general practice. Methods: The RAND/UCLA appropriateness method (a modified Delphi method) was used to define these assigned QIs. First, a literature review was performed to generate a list of possible QIs for HF. Second, by applying the SMART principle, 25 QIs were withheld. Third, an expert panel of health care providers experienced in HF (cardiologists, general practitioners and HF nurses) was convened. Finally, the panellists rated the QIs for appropriateness in three rounds. Results: The withheld QIs highlighted diverse aspects of HF care. In round 1, 20 of the 25 QIs were considered appropriate, and five were considered uncertain. In round 2, 19 QIs were rated appropriate, four inappropriate, and two uncertain. In round 3, the final 19 appropriate QIs were ranked to form a top 10. The top three began with the identification of the aetiology of HF, continued with the differentiation between HF with preserved and reduced ejection fraction, and concluded with the treatment of HF with an ACE-I and a ß-blocker in third place. Conclusion: In this study, 19 QIs for HF in general practice, extractable from the EHR, were identified. These QIs should enable the qualitative monitoring of HF care.

Designing a Patient Portal for Patient-Centered Care: Cross-Sectional Survey.

BACKGROUND: In recent literature, patient portals are considered as important tools for the delivery of patient-centered care. To date, it is not clear how patients would conceptualize a patient portal and which health information needs they have when doing so. OBJECTIVE: This study aimed (1) to investigate health information needs, expectations, and attitudes toward a patient portal and (2) to assess whether determinants, such as patient characteristics, health literacy, and empowerment status, can predict two different variables, namely the importance people attribute to obtaining health information when using a patient portal and the expectations concerning personal health care when using a patient portal. METHODS: We conducted a cross-sectional survey of the Flemish population on what patients prefer to know about their digital health data and their expectations and attitudes toward using a patient portal to access their electronic health record. People were invited to participate in the survey through newsletters, social media, and magazines. We used a questionnaire including demographics, health characteristics, health literacy, patient empowerment, and patient portal characteristics. RESULTS: We received 433 completed surveys. The health information needs included features such as being notified when one's health changes (371/396, 93.7%), being notified when physical parameters increase to dangerous levels (370/395, 93.7%), observing connections between one's symptoms or diseases or biological parameters (339/398, 85.2%), viewing the evolution of one's health in function of time (333/394, 84.5%), and viewing information about the expected effect of treatment (349/395, 88.4%). Almost 90% (369/412) of respondents were interested in using a patient portal. Determinants of patients' attachment for obtaining health information on a patient portal were (1) age between 45 and 54 years (P=.05); (2) neutral (P=.03) or interested attitude (P=.008) toward shared decision making; and (3) commitment to question physicians' decisions (P=.03, R2=0.122). Determinants of patients' expectations on improved health care by accessing a patient portal were (1) lower education level (P=.04); (2) neutral (P=.03) or interested attitude (P=.008) toward shared decision making; and (3) problems in understanding health information (P=.04; R2=0.106). CONCLUSIONS: The interest in using a patient portal is considerable in Flanders. People would like to receive alerts or some form of communication from a patient portal in case they need to act to manage their health. Determinants such as education, attached importance to shared decision making, difficulties in finding relevant health information, and the attached importance in questioning the decisions of physicians need to be considered in the design of a patient portal.