Inter-Rater Agreement of Intensivists Evaluating the Goal Concordance of Preference-Sensitive ICU Interventions.

CONTEXT: Goal-concordant care has been identified as an important outcome of advance care planning and shared decision-making initiatives. However, validated methods for measuring goal concordance are needed. OBJECTIVES: To estimate the inter-rater reliability of senior critical care fellows rating the goal concordance of preference-sensitive interventions performed in intensive care units (ICUs) while considering patient-specific circumstances as described in a previously proposed methodology. METHODS: We identified ICU patients receiving preference-sensitive interventions in three adult ICUs at Johns Hopkins Hospital. A simulated cohort was created by randomly assigning each patient one of 10 sets of goals and preferences about limiting life support. Critical care fellows then independently reviewed patient charts and answered two questions: 1) Is this patient's goal achievable? and 2) Will performing this intervention help achieve the patient's goal? When the answer to both questions was yes, the intervention was rated as goal concordant. Inter-rater agreement was summarized by estimating intraclass correlation coefficient using mixed-effects models. RESULTS: Six raters reviewed the charts of 201 patients. Interventions were rated as goal concordant 22%-92% of the time depending on the patient's goal-limitation combination. Percent agreement between pairs of raters ranged from 59% to 86%. The intraclass correlation coefficient for ratings of goal concordance was 0.50 (95% CI 0.31-0.69) and was robust to patient age, gender, ICU, severity of illness, and lengths of stay. CONCLUSION: Inter-rater agreement between intensivists using a standardized methodology to evaluate the goal concordance of preference-sensitive ICU interventions was moderate. Further testing is needed before this methodology can be recommended as a clinical research outcome.

Operationalizing the Measuring What Matters Spirituality Quality Metric in a Population of Hospitalized, Critically Ill Patients and Their Family Members.

CONTEXT: Measuring What Matters (MWM) quality indicators support measurement of the percentage of patients who have spiritual discussions, if desired. OBJECTIVES: The objective of this study was to 1) determine the ease of, and barriers to, prospectively collecting MWM spirituality quality measure data and 2) further explore the importance of spirituality in a seriously ill, hospitalized population of critically ill patients and their family members. METHODS: Electronic medical record (EMR) review and cross-sectional survey of intensive care unit (ICU) patients and their family members from October to December 2015. Participants were in four adult ICUs totaling 68 beds at a single academic, urban, tertiary care center which has ICU-assigned chaplains and an in-house, 24-hour, on-call chaplain. RESULTS: All patients had a "Spiritual Risk Screen" which included two questions identifying patient religion and whether a chaplain visit was desired. Approximately 2/3 of ICU patients were eligible, and there were 144 respondents (50% female; 57% patient and 43% family member), with the majority being Caucasian or African American (68% and 21%, respectively). Common religious identifications were Christian or no faith tradition (76% and 11%, respectively). Approximately half of patients had an EMR chaplain note although it did not document presence of a "spiritual discussion." No study patients received palliative care consultation. A majority (85%) noted that spirituality was "important to them" and that prevalence remained high across respondent age, race, faith tradition, or admitting ICU. CONCLUSION: Operationalizing the MWM spirituality quality indicator was challenging as elements of a "spiritual screening" or documentation of a "spiritual discussion" were not clearly documented in the EMR. The high prevalence of spirituality among respondents validates the importance of spirituality as a potential quality metric.