Physician Perceptions of Performance Feedback in a Quality Improvement Activity.

Physician performance and peer comparison feedback can affect physician care quality and patient outcomes. This study aimed to understand family physician perspectives of the value of performance feedback in quality improvement (QI) activities. This study analyzed American Board of Family Medicine open-ended survey data collected between 2004 and 2014 from physicians who completed a QI module that provided pre- and post-QI project individual performance data and peer comparisons. Physicians made 3480 comments in response to a question about this performance feedback, which were generally positive in nature (86%). Main themes that emerged were importance of accurate feedback data, enhanced detail in the content of feedback, and ability to customize peer comparison groups to compare performance to peers with similar patient populations or practice characteristics. Meaningful and tailored performance feedback may be an important tool for physicians to improve their care quality and should be considered an integral part of QI project design.

Improving quality of care and guideline adherence for asthma through a group self-assessment module.

BACKGROUND: The quality of care for asthma remains suboptimal. Compliance with guidelines remains low, but improved adherence to guidelines may increase the quality of care. but. We conducted a trial to determine whether group Self-Assessment Module (SAM) activities led by a facilitator and conducted as part of Maintenance of Certification for Family Physicians (MC-FP) would increase knowledge of and adherence to asthma guidelines. METHODS: Participating physicians completed audits of the charts of patients with asthma before and 6 months after a group SAM. Surveys of physicians' knowledge of asthma guidelines were administered immediately before, immediately after, and 6 months after the group SAM. We tested for differences in knowledge of and adherence to guidelines before and after the SAM using χ(2) and t tests. RESULTS: Thirty-eight physicians in Virginia completed the SAM and had complete data. Participants completed more MC-FP activities than other physicians but were comparable in other characteristics. Except for prescribing controller medications for persistent asthma, all other quality measures significantly improved 6 months after the group SAM. Diagnosis by severity improved from 48.3% to 80.2%, and the use of action plans increased from 8.1% to 54.1%. Physicians' knowledge of guidelines improved immediately after the SAM and was sustained at 6 months. Increased knowledge translated into clinical skills: 30% of participants reported comfort with assessing control after the SAM, which increased to 97.5% 6 months after the SAM. CONCLUSIONS: Group SAMs may be an effective method to increase physicians' knowledge of and adherence to clinical guidelines.

Methods for Performing Survival Curve Quality-of-Life Assessments.

BACKGROUND: Many medical decisions involve an implied choice between alternative survival curves, typically with differing quality of life. Common preference assessment methods neglect this structure, creating some risk of distortions. METHODS: Survival curve quality-of-life assessments (SQLA) were developed from Gompertz survival curves fitting the general population's survival. An algorithm was developed to generate relative discount rate-utility (DRU) functions from a standard survival curve and health state and an equally attractive alternative curve and state. A least means squared distance algorithm was developed to describe how nearly 3 or more DRU functions intersect. These techniques were implemented in a program called X-Trade and tested. RESULTS: SQLA scenarios can portray realistic treatment choices. A side effect scenario portrays one prototypical choice, to extend life while experiencing some loss, such as an amputation. A risky treatment scenario portrays procedures with an initial mortality risk. A time trade scenario mimics conventional time tradeoffs. Each SQLA scenario yields DRU functions with distinctive shapes, such as sigmoid curves or vertical lines. One SQLA can imply a discount rate or utility if the other value is known and both values are temporally stable. Two SQLA exercises imply a unique discount rate and utility if the inferred DRU functions intersect. Three or more SQLA results can quantify uncertainty or inconsistency in discount rate and utility estimates. Pilot studies suggested that many subjects could learn to interpret survival curves and do SQLA. LIMITATIONS: SQLA confuse some people. Compared with SQLA, standard gambles quantify very low utilities more easily, and time tradeoffs are simpler for high utilities. When discount rates approach zero, time tradeoffs are as informative and easier to do than SQLA. CONCLUSIONS: SQLA may complement conventional utility assessment methods.

Adding flexible temporal constraints to identify chronic comorbid conditions in ambulatory claims data.

Chronic comorbid conditions are important predictors of primary care outcomes, provide context for clinical decisions, and are potential complications of diseases and treatments. Comorbidity indices and multimorbidity categorization strategies based on administrative claims data enumerate diagnostic codes in easily modifiable lists, but usually have inflexible temporal requirements, such as requiring two claims greater than 30 days apart, or three claims in three quarters. Table structures and claims data search algorithms were developed to support flexible temporal constraints. Tables of disease categories allow subgroups with different numbers of events, different times between similar claims, variable periods of interest, and specified diagnostic code substitutability. The strategy was tested on five years of private insurance claims from 2.2 million working age adults. The contrast between rarely recorded, high prevalence diagnoses (smoking and obesity) and frequently recorded but not necessarily chronic diagnoses (musculoskeletal complaints) demonstrated the advantage of flexible temporal criteria.

Diuretic of choice in ABFM hypertension self-assessment module simulations.

BACKGROUND: Evidence suggests that chlorthalidone has therapeutic advantages over hydrochlorothiazide, perhaps because of a longer antihypertensive effect. Although guidelines such as the Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure treat diuretics as a class, some experts believe chlorthalidone is the diuretic of choice in hypertension management. We evaluated diplomates' use of chlorthalidone and hydrochlorothiazide as first-choice diuretics in virtual patient simulations of hypertension in the American Board of Family Medicine Maintenance of Certification for Family Physicians self-assessment modules. METHODS: We examined action logs for hypertension simulations completed between 2004 and 2011 and identified initial antihypertensive choices made by diplomates. We tabulated uses of any diuretic as initial treatment, distinguishing between chlorthalidone, hydrochlorothiazide, and other diuretic choices. We examined trends in the use of diuretics and chlorthalidone in simulations using linear models. RESULTS: Chlorthalidone use increased 0.7 percentage points per year (test for ß >0, P < .0013), from approximately 1.3% of simulation prescriptions in 2004 to about 4.8% in 2010 and 2011, whereas hydrochlorothiazide prescriptions fell 2 percentage points per year (P = .035), from nearly 57% to 47%. As a fraction of all diuretic use, chlorthalidone increased 1.4 percentage points per year (P = .0006), from 2% to 9%. CONCLUSIONS: Small but growing numbers of diplomates are heeding recommendations in the growing literature to start with chlorthalidone rather than hydrochlorothiazide, at least in the virtual patient environment. Observed choices in virtual patient management strongly imply that this is a valid topic for additional attention in the hypertension self-assessment module.

Family physicians' completion of scoring criteria in Virtual Patient encounters.

The American Board of Family Medicine (ABFM) has used a 60-item Multiple Choice Question (MCQ) section followed by a Virtual Patient (VP) exercise in Maintenance Of Certification (MOC) since 2004, and has had an asthma module since 2005. The original asthma VP criteria anticipated some Expert Panel Report-3 recommendations, such as home peak flow monitoring and a written plan, that were added to the MCQ section only when the guideline was updated in 2007. VP completion rates for these criteria improved markedly with the MCQ update, while other criteria completion rates were stable. Asthma criteria completion rates are not predicted by the strength of evidence for the criteria. User interface details influence criteria completion rates, but did not affect the changes observed in 2007. Asthma MCQ content affects Diplomate performance on asthma VP: this translational step suggests that MOC exercises could result in improved care for real patients.

Variation over time in preventable hospitalization rates across counties.

BACKGROUND: The Agency for Health Care Research and Quality developed 14 prevention quality indicators (PQIs), including four PQIs related to preventable hospitalizations for diabetes and one to asthma. Quality indicators vary across counties, but variation over time has not been described. METHODS: The Kentucky Cabinet for Health and Family Services published PQI data for each county in Kentucky in each of the 3 years from 2006 to 2008. Variation and correlations among PQI measures were calculated. RESULTS: PQI rates often varied 10-fold between counties. Repeated measures of four PQIs were highly correlated, suggesting local health care processes that are stable over time. Some PQIs, such as PQI01--emergent complications of blood glucose control--correlated poorly with other measures. Other PQIs are correlated over geography and time, including PQI03 (long-term complications of diabetes); PQI14 (poorly controlled diabetes); and PQI15 (asthma). CONCLUSIONS: These county PQI measures were stable over time. Stability implies that PQI measures were not the result of random processes and did not rapidly shift. However, some health improvement needs varied between counties. Although tailoring health promotion interventions to each county's needs may be complex, stable needs afford time to undertake targeted quality improvement efforts.

Using county-level public health data to prioritize medical education topics.

INTRODUCTION: Medical education topics might be locally prioritized using public health data on health outcomes and risk factors unrelated to quality of care. METHODS: The Missouri Information for Community Assessment (MICA) supplied preventable hospitalization rates (PHRs) for asthma, chronic obstructive pulmonary disease (COPD), diabetes, heart failure, and hypertension in 114 counties from 1998 to 2002. For each disease, a linear regression model predicted PHR from behavior, access, and disease prevalence data from MICA and other public data sources. For each disease in each county, the residual, unexplained PHR should include effects of local medical practices. Variation in relative priority of diseases between counties was estimated from raw PHR and unexplained PHR. RESULTS: The raw values of the five PHRs varied geographically in different patterns. Regression models explained between 46% and 83% of the variability. The medical education priorities implied by unexplained PHR values differ from priorities inferred from unadjusted PHR or disease prevalence. DISCUSSION: Patient behavior and poor health care access contribute to PHR but do not fully explain variation in PHR. If county-level unexplained PHR values identify high priority medical education topics, then other measures of importance, notably disease prevalence and PHR, are poor identifiers of high value topics. Although available predictor and outcome variables constrain the current analysis, unexplained variation in health outcome measures might identify educational opportunities. These observations suggest strategies for balancing and evaluating controlled trials of knowledge dissemination efforts and eventually for deploying educational activities.

Value of information in virtual patient performance evaluations.

The American Board of Family Medicine (ABFM) uses virtual patients (VP) in Maintenance of Certification. User queries might be evaluated by calculating diagnostic value of information (VOI). Typical primary care queries address concerns far beyond efficient diagnosis. We identify practical constraints on VOI scoring in simulations of primary care practice. We imagine a Bayesian physician as the ideal candidate for VOI scoring. Bayesian physicians require an explicit decision making perspective and many local data.

Value of information in virtual patients portraying pharyngitis.

The American Board of Family Medicine (ABFM) uses virtual patients (VP) in Maintenance of Certification. Theoretically, the ABFM could score users on the value of information (VOI) in their queries. Pharyngitis, a prototypical primary care topic, appears practically intractable to VOI scoring. Problems include conflicting goals of various stakeholders and strong regional and temporal variability in disease characteristics. Nevertheless, the analysis yielded insights that may shift, or validate, the ABFM's selection of scoring criteria.

Modeling relief.

The American Board of Family Medicine deployed virtual patient simulations in 2004 to evaluate Diplomates' diagnostic and management skills. A previously reported dynamic process generates general symptom histories from time series data representing baseline values and reactions to medications. The simulator also must answer queries about details such as palliation and provocation. These responses often describe some recurring pattern, such as, "this medicine relieves my symptoms in a few minutes." The simulator can provide a detail stored as text, or it can evaluate a reference to a second query object. The second query object can generate details using a single Bayesian network to evaluate the effect of each drug in a virtual patient's medication list. A new medication option may not require redesign of the second query object if its implementation is consistent with related drugs. We expect this mechanism to maintain realistic responses to detail questions in complex simulations.

Maintenance of certification for family physicians (MC-FP) self assessment modules (SAMs): the first year.

INTRODUCTION: In 2000, the American Board of Medical Specialties adopted Maintenance of Certification (MOC) to replace intermittent, periodic recertification. MOC consists of 4 components: demonstration of professionalism (part I); commitment to life-long learning (part II); demonstration of cognitive expertise (part III); and evaluation of performance in practice (part IV). The American Board of Family Medicine (ABFM) implemented Maintenance of Certification for Family Physicians (MC-FP) in 2004, with its MC-FP part II self-assessment modules (SAMs) as the focus of the first year's activities. METHODS: The SAMs use materials and resources provided at the ABFM's website (www.theabfm.org). As of April 2005, approximately 7000 Diplomates had successfully completed SAMs in essential hypertension (N = 2351) and type 2 diabetes mellitus (N = 4648). Participants completed categorical modified Likert scale evaluations to receive continuing education credit, and many offered unstructured free-text comments regarding the clinical simulation component. These free-text comments were entered into the AnSWR qualitative analysis program from the Centers for Disease Control and Prevention. Text coding was performed by 2 authors (MDH, DJI). As no inferential analyses or comparisons were anticipated, the authors conducted no studies of inter-rater consistency. Results are reported as means (SD) and medians for continuous data, and as frequencies for count data. RESULTS: Likert-scale ratings indicated generally favorable responses (predominantly 5 to 6 on a 6-point scale) to the hypertension and diabetes SAMs. In addition, over half (ie, 55% for hypertension and 54% for diabetes participants) of the respondents indicated that the experience would lead to changes in their practices. Navigation and system operation issues predominated in the free-text comments offered for the diabetes and hypertension simulations. CONCLUSION: The MC-FP SAMs received generally favorable ratings in the program's first year. The SAMs underwent a number of modifications and improvements during the first year, largely in response to feedback and suggestions from ABFM Diplomates.