Improving Patients' Choice of Clinician by Including Roll-up Measures in Public Healthcare Quality Reports: an Online Experiment.

BACKGROUND: Public reports on healthcare quality typically include complex data. To lower the cognitive burden of interpreting these data, some report designers create summary, or roll-up, measures combining multiple indicators of quality into one score. Little is known about how the availability of roll-ups affects clinician choice. OBJECTIVE: To determine how presenting quality scores at different levels of aggregation affects patients' clinician choices. DESIGN: We conducted a simulated clinician-choice experiment, randomizing participants to three versions of a public reporting website and comparing their clinician choices. One version aggregated all clinician-level quality measures into roll-ups, the second provided disaggregated (drill-down) scores only, and the third offered both roll-ups and drill-downs. PARTICIPANTS: Five hundred fifty panelists drawn from a probability-based Internet panel. MAIN MEASURES: We assessed the amount of effort participants exerted by tracking the length of time spent on the website and the number of concrete actions taken on the website (e.g., clicking items). We evaluated decision quality by measuring whether participants selected a clinician who performed more poorly than others and incongruence between participants' stated preferences for dimensions of quality and their chosen clinician's performance on those dimensions. KEY RESULTS: Participants seeing drill-downs alone (mean = 14.9) or with roll-ups (mean = 19.2) took more actions than those who saw roll-ups alone (mean = 10.5) (ps < 0.05). However, participants seeing only drill-downs made poorer choices than those who saw roll-ups alone or with drill-downs. More participants seeing drill-downs chose a clinician who was outperformed (36.3% versus 23.4% [roll-up] and 25.6% [drill-down + roll-up], ps < 0.05) and made choices incongruent with stated preferences (51.2% versus 45.6% [roll-up] and 47.5% [drill-down + roll-up], ps < 0.05). The distinction between roll-up and drill-down was somewhat stronger for sicker participants. CONCLUSIONS: Our results suggest that roll-ups in healthcare quality reports, alone or as a complement to drill-downs, can help patients make better decisions for themselves.

What Words Convey: The Potential for Patient Narratives to Inform Quality Improvement.

Policy Points Narratives about patients' experiences with outpatient care are essential for quality improvement because they convey ample actionable information that both elaborates on existing domains within patient experience surveys and describes multiple additional domains that are important to patients. The content of narrative feedback from patients can potentially be translated to improved quality in multiple ways: clinicians can learn from their own patients, groups of clinicians can learn from the experience of their peers' patients, and health system administrators can identify and respond to patterns in patients' accounts that reflect systemic challenges to quality. Consistent investment by payers and providers is required to ensure that patient narratives are rigorously collected, analyzed fully, and effectively used for quality improvement. CONTEXT: For the past 25 years, health care providers and health system administrators have sought to improve care by surveying patients about their experiences. More recently, policymakers have acted to promote this learning by deploying financial incentives tied to survey scores. This article explores the potential of systematically elicited narratives about experiences with outpatient care to enrich quality improvement. METHODS: Narratives were collected from 348 patients recruited from a nationally representative Internet panel. Drawing from the literature on health services innovation, we developed a two-part coding schema that categorized narrative content in terms of (a) the aspects of care being described, and (b) the actionability of this information for clinicians, quality improvement staff, and health system administrators. Narratives were coded using this schema, with high levels of reliability among the coders. FINDINGS: The scope of outpatient narratives divides evenly among aspects of care currently measured by patient experience surveys (35% of content), aspects related to measured domains but not captured by existing survey questions (31%), and aspects of care that are omitted from surveys entirely (34%). Overall, the narrative data focused heavily on relational aspects of care (43%), elaborating on this aspect of experience well beyond what is captured with communication-related questions on existing surveys. Three-quarters of elicited narratives had some actionable content, and almost a third contained three or more separate actionable elements. CONCLUSIONS: In a health policy environment that incentivizes attention to patient experience, rigorously elicited narratives hold substantial promise for improving quality in general and patients' experiences with care in particular. They do so in two ways: by making concrete what went wrong or right in domains covered by existing surveys, and by expanding our view of what aspects of care matter to patients as articulated in their own words and thus how care can be made more patient-centered. Most narratives convey experiences that are potentially actionable by those committed to improving health care quality in outpatient settings.

With Small Power, Comes Great Responsibility: Lessons Learned from an Evaluation of Veteran and Military Mental Health Public Awareness Campaigns.

This study was conducted to determine the feasibility of conducting a cost-benefit evaluation of federally-funded media campaigns encouraging mental health help-seeking among United States military personnel and veterans. To calculate the necessary sample size for the evaluation, we obtained campaign costs, and determined the number of treatment seekers needed for the campaign to break even with its cost and the associated population change that an evaluation would need to detect. The sample size needed for an evaluation with 80% power was greater than the total population of U.S. military personnel and veterans. Given that the necessary sample size exceeds the population to be sampled, an appropriately powered outcome evaluation is not feasible. Other programs that would be cost effective with extremely small effect sizes should not be subject to underpowered and thus inaccurate empirical outcome evaluation.

Effects of contact-based mental illness stigma reduction programs: age, gender, and Asian, Latino, and White American differences.

PURPOSE: Mental illness stigma disproportionately affects help seeking among youth, men, and ethnic minorities. As part of a comprehensive statewide initiative to reduce mental illness stigma and discrimination in California, a broad set of contact-based educational programs were widely disseminated. This study examined whether the effects of contact-based educational programs varied depending on the age, gender, and race-ethnicity of participants. METHODS: Participants (N = 4122) attended a contact-based educational program that was delivered as part of the statewide initiative to reduce mental illness stigma and discrimination. Self-administered surveys assessing beliefs, attitudes, and intentions toward mental illnesses and treatment were conducted immediately before and after participation in contact-based educational programs. RESULTS: Participant age, gender, and race-ethnicity significantly moderated pre-post changes in mental illness stigma. Although all groups exhibited significant pre-post changes across most of the stigma domains assessed, young adults, females, and Asian and Latino American participants reported larger improvements compared to older adults, males, and Whites, respectively. CONCLUSIONS: Findings suggest that contact-based educational programs can achieve immediate reductions in mental illness stigma across a variety of sociodemographic groups and may particularly benefit young adults and racial-ethnic minorities. Further research is needed to assess whether contact-based educational programs can sustain longer-term changes and aid in the reduction of disparities in mental illness stigma and treatment.

Development of the PROMIS health expectancies of smoking item banks.

INTRODUCTION: Smokers' health-related outcome expectancies are associated with a number of important constructs in smoking research, yet there are no measures currently available that focus exclusively on this domain. This paper describes the development and evaluation of item banks for assessing the health expectancies of smoking. METHODS: Using data from a sample of daily (N = 4,201) and nondaily (N = 1,183) smokers, we conducted a series of item factor analyses, item response theory analyses, and differential item functioning analyses (according to gender, age, and race/ethnicity) to arrive at a unidimensional set of health expectancies items for daily and nondaily smokers. We also evaluated the performance of short forms (SFs) and computer adaptive tests (CATs) to efficiently assess health expectancies. RESULTS: A total of 24 items were included in the Health Expectancies item banks; 13 items are common across daily and nondaily smokers, 6 are unique to daily, and 5 are unique to nondaily. For both daily and nondaily smokers, the Health Expectancies item banks are unidimensional, reliable (reliability = 0.95 and 0.96, respectively), and perform similarly across gender, age, and race/ethnicity groups. A SF common to daily and nondaily smokers consists of 6 items (reliability = 0.87). Results from simulated CATs showed that health expectancies can be assessed with good precision with an average of 5-6 items adaptively selected from the item banks. CONCLUSIONS: Health expectancies of smoking can be assessed on the basis of these item banks via SFs, CATs, or through a tailored set of items selected for a specific research purpose.

Development of the PROMIS negative psychosocial expectancies of smoking item banks.

INTRODUCTION: Negative psychosocial expectancies of smoking include aspects of social disapproval and disappointment in oneself. This paper describes analyses conducted to develop and evaluate item banks for assessing psychosocial expectancies among daily and nondaily smokers. METHODS: Using data from a sample of daily (N = 4,201) and nondaily (N =1,183) smokers, we conducted a series of item factor analyses, item response theory analyses, and differential item functioning analyses (according to gender, age, and race/ethnicity) to arrive at a unidimensional set of psychosocial expectancies items for daily and nondaily smokers. We also evaluated performance of short forms (SFs) and computer adaptive tests (CATs) to efficiently assess psychosocial expectancies. RESULTS: A total of 21 items were included in the Psychosocial Expectancies item banks: 14 items are common across daily and nondaily smokers, 6 are unique to daily, and 1 is unique to nondaily. For both daily and nondaily smokers, the Psychosocial Expectancies item banks are strongly unidimensional, highly reliable (reliability = 0.95 and 0.93, respectively), and perform similarly across gender, age, and race/ethnicity groups. A SF common to daily and nondaily smokers consists of 6 items (reliability = 0.85). Results from simulated CATs showed that, on average, fewer than 8 items are needed to assess psychosocial expectancies with adequate precision when using the item banks. CONCLUSIONS: Psychosocial expectancies of smoking can be assessed on the basis of these item banks via the SF, by using CAT, or through a tailored set of items selected for a specific research purpose.

Greater benefit of self-affirmation for prevention-focused individuals prior to threatening health messages.

OBJECTIVE: Individuals are often defensive toward health messages that suggest they are putting their health at risk because such messages threaten their self-competence and integrity. Although self-affirmation can facilitate prevention behaviors in response to health messages, effects are variable. We examined whether disease prevention focus might strengthen self-affirmation's effects in response to disease prevention messages, given that prevention-focused individuals are likeliest to be persuaded by those messages after self-affirmation attenuates defensiveness. DESIGN: In Study 1, participants were self-affirmed before a message about sexually transmitted infections. In Studies 2 and 3, individuals were self-affirmed prior to a message about alcohol and cancer risk. MAIN OUTCOME MEASURES: Studies assessed intentions to use condoms, intentions to reduce alcohol, and willingness to drink alcohol in specific scenarios. RESULTS: In Study 1, self-affirmation facilitated condom use intentions among those higher in prevention focus. In Studies 2 and 3, self-affirmation facilitated lower willingness to consume alcohol among those high in prevention focus. A meta-analysis across the three studies indicated that self-affirmation improved intentions and willingness under high, but not low, prevention focus (d = 0.20, p = .003). CONCLUSION: These findings demonstrate that health prevention-focus can strengthen self-affirmation's effects, thereby improving responsiveness to health communications about behaviors that increase disease risk.

A Rigorous Approach to Large-Scale Elicitation and Analysis of Patient Narratives.

Patient narratives have emerged as promising vehicles for making health care more responsive by helping clinicians to better understand their patients' expectations, perceptions, or concerns and encouraging consumers to engage with information about quality. A growing number of websites incorporate patients' comments. But existing comments have fragmentary content, fail to represent less vocal patients, and can be manipulated to "manage" providers' reputations. In this article, we offer the first empirical test of the proposition that patient narratives can be elicited rigorously and reliably using a five-question protocol that can be incorporated into large-scale patient experience surveys. We tested whether elicited narratives about outpatient care are complete (report all facets of patient experience), balanced (convey an accurate mix of positive and negative events), meaningful (have a coherent storyline), and representative (draw fulsome narratives from all relevant subsets of patients). The tested protocol is strong on balance and representativeness, more mixed on completeness and meaningfulness.

Evaluating an Operator Physical Fitness Test Prototype for Tactical Air Control Party and Air Liaison Officers: A Preliminary Analysis of Test Implementation.

The U.S. Air Force asked the RAND Corporation to assist its development and validation of gender-neutral tests and standards for battlefield airmen (BA) specialties. The Air Force has conducted an extensive validation study of occupational relevance of physical fitness tests and standards. Following the fitness test validation study, one enlisted specialty (Tactical Air Control Party [TACP]) and one officer BA specialty (Air Liaison Officer [ALO]) moved forward with an implementation plan to further evaluate a set of recommended tests and continuation standards. This study describes RAND's assistance to the Air Force on two fronts: (1) conducting a preliminary evaluation of potential issues and concerns that might influence implementation effectiveness and (2) developing a framework for evaluating the implementation of occupationally relevant and specific tests and standards. This work provides the foundation for ongoing review and evaluation of Air Force fitness tests and standards, which are designed to ensure that airmen are capable of performing critical physical tasks associated with their assigned specialties.