Online Forums as a Tool for Broader Inclusion of Voices on Health Care Communication Experiences and Serious Illness Care: Mixed Methods Study.

BACKGROUND: Existing health care research, including serious illness research, often underrepresents individuals from historically marginalized communities. Capturing the nuanced perspectives of individuals around their health care communication experiences is difficult. New research strategies are needed that increase engagement of individuals from diverse backgrounds. OBJECTIVE: The aim of this study was to develop a mixed methods approach with qualitative online forums to better understand health communication experiences of individuals, including people from groups historically marginalized such as Black and Latino individuals; older adults; and people with low income, disability, or serious illness. METHODS: We used a multiphase mixed methods, community-informed research approach to design study instruments and engage participants. We engaged a diverse group of collaborators with lived experience of navigating the health care system who provided feedback on instruments, added concepts for testing, and offered guidance on creating a safe experience for participants (phase 1). We conducted a national quantitative survey between April and May 2021 across intrapersonal, interpersonal, and systems-level domains, with particular focus on interpersonal communication between patients and clinicians (phase 2). We conducted two asynchronous, qualitative online forums, a technique used in market research, between June and August 2021, which allowed us to contextualize the learnings and test concepts and messages (phase 3). Using online forums allowed us to probe more deeply into results and hypotheses from the survey to better understand the "whys" and "whats" that surfaced and to test public messages to encourage action around health. RESULTS: We engaged 46 community partners, including patients and clinicians from a Federally Qualified Health Center, to inform study instrument design. In the quantitative survey, 1854 adults responded, including 50.5% women, 25.2% individuals over 65 years old, and 51.9% individuals with low income. Nearly two-thirds identified as non-Hispanic white (65.7%), 10.4% identified as non-Hispanic Black, and 15.5% identified as Hispanic/Latino. An additional 580 individuals participated in online forums, including 60.7% women, 17.4% individuals over 65 years old, and 49.0% individuals with low income. Among the participants, 70.3% identified as non-Hispanic white, 16.0% as non-Hispanic Black, and 9.5% as Hispanic/Latino. We received rich, diverse input from our online forum participants, and they highlighted satisfaction and increased knowledge with engagement in the forums. CONCLUSIONS: We achieved modest overrepresentation of people who were over 65 years old, identified as non-Hispanic Black, and had low income in our online forums. The size of the online forums (N=580) reflected the voices of 93 Black and 55 Hispanic/Latino participants. Individuals who identify as Hispanic/Latino remained underrepresented, likely because the online forums were offered only in English. Overall, our findings demonstrate the feasibility of using the online forum qualitative approach in a mixed methods study to contextualize, clarify, and expound on quantitative findings when designing public health and clinical communications interventions.

Training Novice Raters to Assess Nontechnical Skills of Operating Room Teams.

OBJECTIVE: To our knowledge, no curricula have been described for training novice, nonclinician raters of nontechnical skills in the operating room (OR). We aimed to report the reliability of Oxford Non-Technical Skills (NOTECHS) ratings provided by novice raters who underwent a scalable curriculum for learning to assess nontechnical skills of OR teams. DESIGN: In-person training course to apply the NOTECHS framework to assessing OR teams' nontechnical skill performance, led by 2 facilitators and involving 5 partial-day sessions of didactic presentations, video simulation, and live OR observation with postassessment debriefing. NOTECHS ratings were submitted after each of 11 video scenarios and 8 live operations for the total NOTECHS team rating (including surgical/anesthesiology/nursing subteams) and for each NOTECHS skill category-situation awareness, problem solving and decision making, teamwork and cooperation, leadership and management. Inter-rater reliability was determined by calculating the intraclass correlation coefficient (ICC, range 0-1). SETTING: Training for outcome measurement during a quality improvement initiative focused on surgical safety in 3 public hospitals in Singapore. Two trainings were conducted in May 2019 and January 2020. PARTICIPANTS: Ten novice raters who were existing hospital staff and had overall minimal OR experience and no prior experience with nontechnical skill assessment. RESULTS: ICC for the total NOTECHS team rating was 0.89 (95% confidence interval [CI], 0.87-0.91). ICCs for each NOTECHS category were as follows: situation awareness, 0.83 (95% CI, 0.78-0.88); problem solving and decision-making, 0.76 (95% CI, 0.70-0.83); teamwork and cooperation, 0.84 (95% CI, 0.79-0.88); leadership and management, 0.81 (95% CI, 0.75-0.86). CONCLUSIONS: This training curriculum for nontechnical skill assessments of OR teams was associated with high inter-rater reliability from novice raters with minimal collective OR experience. Using scalable training materials to produce reliable measurements of OR team performance, this nontechnical skills assessment curriculum may contribute to future QI projects aimed at improving surgical safety.

Serious Illness Conversations With Outpatient Oncology Clinicians: Understanding the Patient Experience.

PURPOSE: Guidelines recommend earlier advance care planning discussions focused on goals and values (serious illness communication) among oncology patients. We conducted a prospective, cross-sectional quality improvement evaluation of patients who had a serious illness conversation (SIC) with an oncology clinician using the Serious Illness Conversation Guide to understand patient perceptions of conversations using a structured guide. METHODS: We contacted 66 oncology patients with an SIC documented in the electronic health record. Thirty-two patients (48%) responded to survey and/or structured interview questions by telephone. We used summary statistics and thematic analysis to analyze results. RESULTS: Twenty-eight respondents (90%) reported that the SIC was worthwhile. Seventeen respondents (55%) reported that the conversation increased their understanding of their future health, and 18 (58%) reported that the conversation increased their sense of closeness with their clinician. Although the majority of respondents (28 [90%]) reported that the conversation increased (13 [42%]) or had no effect (15 [48%]) on their hopefulness, a small minority (3 [10%]) reported a decrease in hopefulness. Qualitative analysis revealed 6 themes: clinician-patient relationship, impact on well-being, memorable characteristics of the conversation, improved prognostic understanding, practical planning, and family communication. CONCLUSION: SICs are generally acceptable to oncology patients (nonharmful to the vast majority, positive for many). Our qualitative analysis suggests a positive impact on prognostic understanding and end-of-life planning, but opportunities for improvement in the delivery of prognosis and preparing patients for SICs. Our data also identify a small cohort who responded negatively, highlighting an important area for future study.

Measuring Goal-Concordant Care: Results and Reflections From Secondary Analysis of a Trial to Improve Serious Illness Communication.

CONTEXT: Many consider goal-concordant care (GCC) to be the most important of advance care planning and palliative care. Researchers face significant challenges in attempting to measure this outcome. We conducted a randomized controlled trial to assess the effects of a system-level intervention to improve serious illness communication on GCC and other outcomes. OBJECTIVES: To describe our measurement approach to GCC, present findings from a post-hoc analysis of trial data, and discuss lessons learned about measuring GCC. METHODS: Using trial data collected to measure GCC, we analyzed ratings and rankings from a nonvalidated survey of patient priorities in the setting of advanced cancer, the Life Priorities Scale, and compared outcomes with correlative measures. RESULTS: Participants commonly rated several predetermined and literature-derived priorities as important but did so in ways that were commonly incongruent with rankings. Ratings were frequently stable over time; rankings less so. Rankings are more likely to help assess the degree to which care is goal concordant but may be best augmented by corollary measures that signal achievement of a given priority. CONCLUSION: Measuring GCC remains a fundamental challenge to palliative care researchers. Ratings attest to the fact that many things matter to patients; however, rankings can better determine what matters most. Insights gained from our experience may guide future research aiming to use this outcome to assess the effect of intervention to improve serious illness care.

Effect of the Serious Illness Care Program on Health Care Utilization at the End of Life for Patients with Cancer.

Objectives: To determine the effect of the Serious Illness Care Program on health care utilization at the end of life in oncology. Design: Analysis of the secondary outcome of health care utilization as part of a cluster-randomized clinical trial that ran from 2012 to 2016. Clinicians in the intervention group received training, coaching, and system supports to have discussions with patients using a Serious Illness Conversation Guide (SICG); clinicians in the control arm followed usual care. Setting/Subject: Patients with advanced cancer who died within two years of enrollment at the Dana-Farber Cancer Institute. Measurement: Health care utilization was abstracted from the electronic medical record using the National Quality Forum (NQF)-endorsed indicators of aggressive cancer care at the end of life and scored from 0 to 6 (one point for each aggressive indicator); t tests and chi-square tests were used to determine differences between intervention and control patients. Results: The charts of 159 patients who died were reviewed. Neither the main outcome of mean number of aggressive indicators (0.9 vs. 0.9, p = 0.84) nor the proportion of patients with any aggressive care (49% intervention [95% CI: 40-57] vs. 54% control [95% CI: 42-67]) differed between patients in the intervention and control groups. Conclusion: In this analysis of a secondary outcome from a randomized clinical trial of the Serious Illness Care Program, intervention and control patients had similar end-of-life health care utilization as measured by the mean number of NQF-endorsed indicators. Future research efforts should focus on studying the strategies by which communication about patients' prognosis, values, and goals leads to personalized care plans.

Training Clinicians in Serious Illness Communication Using a Structured Guide: Evaluation of a Training Program in Three Health Systems.

Background: Failure to initiate discussions about patients' values and goals in serious illness remains a common problem. Many clinicians are inadequately trained for these discussions. Objective: Evaluate whether a novel train-the-trainer model results in high-quality training that improves clinicians' self-reported competencies in serious illness communication. Design: Multimethod evaluation of an educational program. Setting/Context: In 2016, three faculty at Ariadne Labs (AL) conducted three train-the-trainer courses to equip faculty trainers at each of the three institutions to teach serious illness communication to clinicians. Measures: As collected by a post-training questionnaire, primary evaluation measure is clinicians' self-reported change in skills after the training compared with before. Secondary measures include a course evaluation and qualitative learnings. Results: From 2016 to 2018, AL trained 22 trainers (19/22 were palliative care specialists) in three systems, who trained 297 clinicians (49% physicians; 35% advanced practice clinicians; 12% registered nurses, social workers, or chaplain; 4.0% Other) spanning subspecialties (48%); primary care (28%); palliative care (17%); and other (7.1%). Clinicians reported statistically significant improvement in all skills for two of the systems, with a third system demonstrating improvement in all skills with two reaching statistical significance (p < 0.0001). Participants rated the quality of the training highly (95% mostly/extremely effective) and shared a diverse array of takeaways that reflect positive shifts in knowledge, attitudes, and skills. Conclusion: Serious illness communication training, delivered through a train-the-trainer model, was highly acceptable and resulted in significant self-reported improvements in competencies of clinicians. This may be a viable method for health systems seeking to train their clinical workforce.

Evaluating an Intervention to Improve Communication Between Oncology Clinicians and Patients With Life-Limiting Cancer: A Cluster Randomized Clinical Trial of the Serious Illness Care Program.

IMPORTANCE: Earlier clinician-patient conversations about patients' values, goals, and preferences in serious illness (ie, serious illness conversations) are associated with better outcomes but occur inconsistently in cancer care. OBJECTIVE: To evaluate the efficacy of a communication quality-improvement intervention in improving the occurrence, timing, quality, and accessibility of documented serious illness conversations between oncology clinicians and patients with advanced cancer. DESIGN, SETTING, PARTICIPANTS: This cluster randomized clinical trial in outpatient oncology was conducted at the Dana-Farber Cancer Institute and included physicians, advanced-practice clinicians, and patients with cancer who were at high risk of death. MAIN OUTCOMES AND MEASURES: The primary outcomes (goal-concordant care and peacefulness at the end of life) are published elsewhere. Secondary outcomes are reported herein, including (1) documentation of at least 1 serious illness conversation before death, (2) timing of the initial conversation before death, (3) quality of conversations, and (4) their accessibility in the electronic medical record (EMR). RESULTS: We enrolled 91 clinicians (48 intervention, 43 control) and 278 patients (134 intervention, 144 control). Of enrolled patients, 58% died during the study (n=161); mean age was 62.3 years (95% CI, 58.9-65.6 years); 55% were women (n=88). These patients were cared for by 76 of the 91 enrolled clinicians (37 intervention, 39 control); years in practice, 11.5 (95% CI, 9.2-13.8); 57% female (n=43). Medical record review after patients' death demonstrated that a significantly higher proportion of intervention patients had a documented discussion compared with controls (96% vs 79%, P = .005) and intervention conversations occurred a median of 2.4 months earlier (median, 143 days vs 71 days, P < .001). Conversation documentation for intervention patients was significantly more comprehensive and patient centered, with a greater focus on values or goals (89% vs 44%, P < .001), prognosis or illness understanding (91% vs 48%, P < .001), and life-sustaining treatment preferences (63% vs 32%, P = .004). Documentation about end-of-life care planning did not differ between arms (80% intervention vs 68% control, P = .08). Significantly more intervention patients had documentation that was accessible in the EMR (61% vs 11%, P < .001). CONCLUSIONS AND RELEVANCE: This communication quality-improvement intervention resulted in more, earlier, better, and more accessible serious illness conversations documented in the EMR. To our knowledge, this is the first such study to demonstrate improvement in all 4 of these outcomes. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT01786811.

Effect of the Serious Illness Care Program in Outpatient Oncology: A Cluster Randomized Clinical Trial.

Importance: High-quality conversations between clinicians and seriously ill patients about values and goals are associated with improved outcomes but occur infrequently. Objective: To examine feasibility, acceptability, and effect of a communication quality-improvement intervention (Serious Illness Care Program) on patient outcomes. Design, Setting, and Participants: A cluster randomized clinical trial of the Serious Illness Care Program in an outpatient oncology setting was conducted. Patients with advanced cancer (n = 278) and oncology clinicians (n = 91) participated between September 1, 2012, and June 30, 2016. Data analysis was performed from September 1, 2016, to December 27, 2018. All analyses were conducted based on intention to treat. Interventions: Tools, training, and system changes. Main Outcomes and Measures: The coprimary outcomes included goal-concordant care (Life Priorities) and peacefulness (Peace, Equanimity, and Acceptance in the Cancer Experience questionnaire) at the end of life. Secondary outcomes included therapeutic alliance (Human Connection Scale), anxiety (Generalized Anxiety Disorder 7 scale), depression (Patient Health Questionnaire 9), and survival. Uptake and effectiveness of clinician training, clinician use of the conversation tool, and conversation duration were evaluated. Results: Data from 91 clinicians in 41 clusters (72.9% participation; intervention, n = 48; control, n = 43; 52 [57.1%] women) and 278 patients (45.8% participation; intervention, n = 134; control, n = 144; 148 [53.2%] women) were analyzed. Forty-seven clinicians (97.9%) rated the training as effective (mean [SD] score, 4.3 [0.7] of 5.0 possible); of 39 who received a reminder, 34 (87.2%) completed at least 1 conversation (median duration, 19 minutes; range, 5-70). Peacefulness, therapeutic alliance, anxiety, and depression did not differ at baseline. The coprimary outcomes were evaluated in 64 patients; no significant differences were found between the intervention and control groups. However, the trial demonstrated significant reductions in the proportion of patients with moderate to severe anxiety (10.2% vs 5.0%; P = .05) and depression symptoms (20.8% vs 10.6%; P = .04) in the intervention group at 14 weeks after baseline. Anxiety reduction was sustained at 24 weeks (10.4% vs 4.2%; P = .02), but depression reduction was not sustained (17.8% vs 12.5%; P = .31). Survival and therapeutic alliance did not differ between groups. Conclusions and Relevance: The results of this cluster randomized clinical trial were null with respect to the coprimary outcomes of goal-concordant care and peacefulness at the end of life. Methodologic challenges for the primary outcomes, including measure selection and sample size, limit the conclusions that can be drawn from the study. However, the significant reductions in anxiety and depression in the intervention group are clinically meaningful and require further study. Trial Registration: ClinicalTrials.gov identifier: NCT01786811.

A Qualitative Study of Serious Illness Conversations in Patients with Advanced Cancer.

Background: Conversations with seriously ill patients about their values and goals have been associated with reduced distress, a better quality of life, and goal-concordant care near the end of life. Yet, little is known about how such conversations are conducted. Objective: To characterize the content of serious illness conversations and identify opportunities for improvement. Design: Qualitative analysis of audio-recorded, serious illness conversations using an evidence-based guide and obtained through a cluster randomized controlled trial in an outpatient oncology setting. Setting/Measurements: Clinicians assigned to the intervention arm received training to use the "Serious Illness Conversation Guide" to have a serious illness conversation about values and goals with advanced cancer patients. Conversations were de-identified, transcribed verbatim, and independently coded by two researchers. Key themes were analyzed. Results: A total of 25 conversations conducted by 16 clinicians were evaluated. The median conversation duration was 14 minutes (range 4-37), with clinicians speaking half of the time. Thematic analyses demonstrated five key themes: (1) supportive dialogue between patients and clinicians; (2) patients' openness to discuss emotionally challenging topics; (3) patients' willingness to articulate preferences regarding life-sustaining treatments; (4) clinicians' difficulty in responding to emotional or ambiguous patient statements; and (5) challenges in discussing prognosis. Conclusions: Data from this exploratory study suggest that seriously ill patients are open to discussing values and goals with their clinician. Yet, clinicians may struggle when disclosing a time-based prognosis and in responding to patients' emotions. Such skills should be a focus for additional training for clinicians caring for seriously ill patients.

A Descriptive Analysis of End-of-Life Conversations With Long-Term Glioblastoma Survivors.

BACKGROUND: Early, high-quality serious illness (SI) conversations are critical for patients with glioblastoma (GBM) but are often mistimed or mishandled. OBJECTIVE: To describe the prevalence, timing, and quality of documented SI conversations and evaluate their focus on patient goals/priorities. DESIGN/PARTICIPANTS: Thirty-three patients with GBM enrolled in the control group of a randomized controlled trial of a communication intervention and were followed for 2 years or until death. At baseline, all patients answered a validated question about preferences for life-extending versus comfort-focused care and completed a Life Priorities Survey about their goals/priorities. In this secondary analysis, retrospective chart review was performed for 18 patients with GBM who died. Documented SI conversations were systematically identified and evaluated using a codebook reflecting 4 domains: prognosis, goals/priorities, end-of-life planning, and life-sustaining treatments. Patient goals/priorities were compared to documentation. MEASUREMENTS/RESULTS: At baseline, 16 of 24 patients preferred life-extending care. In the Life Priorities Survey, goals/priorities most frequently ranked among the top 3 were "Live as long as possible," "Be mentally aware," "Provide support for family," "Be independent," and "Be at peace." Fifteen of 18 patients had at least 1 documented SI conversation (range: 1-4). Median timing of the first documented SI conversation was 84 days before death (range: 29-231; interquartile range: 46-119). Fifteen patients had documentation about end-of-life planning, with "hospice" and "palliative care" most frequently documented. Five of 18 patients had documentation about their goals. CONCLUSION: Patients with GBM had multiple goals/priorities with potential treatment implications, but documentation showed SI conversations occurred relatively late and infrequently reflected patient goals/priorities.

A family psychosocial risk questionnaire for use in pediatric practice.

The objective of this study is to develop new methods to better identify psychosocial risk such that children with the greatest risk of poor future outcomes receive more intensive preventive health services. Based on structured literature review and secondary data analysis, a 52-item psychosocial risk questionnaire was administered to 2,083 families of children (<36 months). To quantify the questionnaire's construct validity, developmental concern was assessed with the Ages and Stages Questionnaire version II (ASQ) [n = 1,163]. An iterative model selection process was used to produce the most parsimonious predictive model. Model fit was examined using c-statistics, the Hosmer-Lemeshow test, and a heuristic measure of model overfit based on the fitted log-likelihood values and associated number of degrees of freedom. We found 13 items easily obtained from parental report produced a regression model with a c-statistic of 0.70. Using an integer scoring system derived from the regression model, we calculated stratum specific likelihood ratios to revise a given prior probability of ASQ failure. The posterior probability of ASQ failure was 44.9 % for a child in the highest risk group (score >25) on the questionnaire, more than double our observed average failure rate of 19.5 %, while it was less than 7 % for a child with the lowest possible score on the questionnaire. Thirteen parent-reported items can be compiled into a summary psychosocial risk questionnaire that predicts failure on developmental screening among preschool children. With further validation, this questionnaire could conceivably be used by clinicians to tailor pediatric preventive care to children at varying levels of risk.

Effectiveness of developmental screening in an urban setting.

OBJECTIVE: To determine the effectiveness of developmental screening on the identification of developmental delays, early intervention (EI) referrals, and EI eligibility. METHODS: This randomized controlled, parallel-group trial was conducted from December 2008 to June 2010 in 4 urban pediatric practices. Children were eligible if they were <30 months old, term, without congenital malformations or genetic syndromes, not in foster care, and not enrolled in EI. Children were randomized to receive 1 of the following: (1) developmental screening using Ages and Stages Questionnaire-II (ASQ-II and Modified Checklist for Autism in Toddlers (M-CHAT) with office staff assistance, (2) developmental screening using ASQ-II and M-CHAT without office staff assistance, or (3) developmental surveillance using age-appropriate milestones at well visits. Outcomes were assessed using an intention-to-treat analysis. RESULTS: A total of 2103 children were enrolled. Most were African-American with family incomes less than $30,000. Children in either screening arm were more likely to be identified with delays (23.0% and 26.8% vs 13.0%; P < .001), referred to EI (19.9% and 17.5% vs 10.2%; P < .001), and eligible for EI services (7.0% and 5.3% vs 3.0%; P < .001) than children in the surveillance arm. Children in the screening arms incurred a shorter time to identification, EI referral, and EI evaluation than children in the surveillance arm. CONCLUSIONS: Children who participated in a developmental screening program were more likely to be identified with developmental delays, referred to EI, and eligible for EI services in a timelier fashion than children who received surveillance alone. These results support policies endorsing developmental screening.

Reading level of Medicaid renewal applications.

OBJECTIVE: To examine compliance of Medicaid-renewal applications to established state guidelines for reading level of Medicaid-related materials. METHODS: We assessed the reading level of the 2008 Medicaid renewal applications by using 3 readability tests: Flesch-Kincaid Grade Level Index, New Fog Count, and FORCAST. RESULTS: In 2008, 45 states and the District of Columbia had reading level guidelines for Medicaid-related materials. Of these, 24 (52.2%) states' Medicaid renewal applications failed to meet their guidelines on all 3 readability tests; in particular, 41 states (89.1%) failed the Flesch-Kincaid Grade Level Index. Only one state's application had a Flesch-Kincaid score below a 5th-grade reading level. CONCLUSIONS: As health care reform unfolds, complying with established reading level guidelines for Medicaid-related materials is one simplification strategy that should be implemented to improve access for Medicaid-eligible families and prevent eligible children from losing coverage unnecessarily.

Influence of maternal health literacy on child participation in social welfare programs: the Philadelphia experience.

We examined the influence of maternal health literacy on child participation in social welfare programs. In this cohort, 20% of the mothers had inadequate or marginal health literacy. Initially, more than 50% of the families participated in Temporary Assistance for Needy Families (TANF), the Food Stamp Program, and Special Supplemental Nutrition Program for Women, Infants, and Children, whereas fewer than 15% received child care subsidies or public housing. In multivariate regression, TANF participation was more than twice as common among children whose mothers had adequate health literacy compared with children whose mothers had inadequate health literacy.