Challenges in Measuring Applied Cognition: Measurement Properties and Equivalence of the Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test (FAMCAT) Applied Cognition Item Bank.

OBJECTIVE: To present challenges in assessment of applied cognition and the results of differential item functioning (DIF) analyses used to inform the development of a computerized adaptive test (CAT). DESIGN: Measurement evaluation cohort study. DIF analyses of 107 items were conducted across educational, age, and sex groups. DIF hypotheses informed the evaluation of the results. SETTING: Hospital-based rehabilitation from a single hospital system. PARTICIPANTS: A total of 2216 hospitalized patients (N=2216). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Applied cognition item pool from multiple sources. RESULTS: Many items were hypothesized to show DIF, particularly for age. Information was moderately high in the lower (cognitive disability) tail of the distribution, but some items were not informative. Reliability estimates were high (>0.89) across all studied groups, regardless of estimation method. There were 35 items with DIF of high magnitude and 19 with accompanying supportive hypotheses. CONCLUSIONS: A key clinical tool in inpatient rehabilitation medicine is assessment of applied functional cognitive ability to inform patient-centered rehabilitation strategies to improve function. This was the first study to evaluate measurement equivalence of the applied cognition item pool across large samples of hospitalized patients. Although about one-third of the item pool evidenced DIF or low discrimination, results supported placement of most items into the bank and its use across groups differing in education, age, and sex. Six items were classified with salient DIF, defined as consistent DIF of high magnitude and or impact, with confirmatory directional DIF hypotheses, generated by content experts. These were recommended for adjustment or removal from the bank; 4 were deleted from the bank and 2 had lowered CAT exposure (administration frequency) rates. Many items hypothesized to show DIF contained content measuring constructs other than applied cognition such as physical frailty, perceptual difficulties, or skills reflective of greater educational attainment. Challenges in measurement of this construct are discussed.

Examination of the Measurement Equivalence of the Functional Assessment in Acute Care MCAT (FAMCAT) Mobility Item Bank Using Differential Item Functioning Analyses.

OBJECTIVE: To assess differential item functioning (DIF) in an item pool measuring the mobility of hospitalized patients across educational, age, and sex groups. DESIGN: Measurement evaluation cohort study. Content experts generated DIF hypotheses to guide the interpretation. The graded response item response theory (IRT) model was used. Primary DIF tests were Wald statistics; sensitivity analyses were conducted using the IRT ordinal logistic regression procedure. Magnitude and impact were evaluated by examining group differences in expected item and scale score functions. SETTING: Hospital-based rehabilitation. PARTICIPANTS: Hospitalized patients (N=2216). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: A total of 111 self-reported mobility items. RESULTS: Two linking items among those used to set the metric across forms evidenced DIF for sex and age: "difficulty climbing stairs step-over-step without a handrail (alternating feet)" and "difficulty climbing 3-5 steps without a handrail." Conditional on the mobility state, the items were more difficult for women and older people (aged ≥65y). An additional 18 items were identified with DIF. Items with both high DIF magnitude and hypotheses related to age were difficulty "crossing road at a 4-lane traffic light with curbs," "jumping/landing on one leg," "strenuous activities," and "descending 3-5 steps with no handrail." Although DIF of higher magnitude was observed for several items, the scale-level effect was relatively small and the exposure rate for the most problematic items was low (0.35, 0.27, and 0.20). CONCLUSIONS: This was the first study to evaluate measurement equivalence of the hospital-based rehabilitation mobility item bank. Although 20 items evidenced high magnitude DIF, 5 of which were related to stairs, the scale-level effect was minimal; however, it is recommended that such items be avoided in the development of short-form measures. No items with salient DIF were removed from calibrations, supporting the use of the item bank across groups differing in education, age, and sex. The bank may thus be useful to assist clinical assessment and decision-making regarding risk for specific mobility restrictions at discharge as well as identifying mobility-related functions targeted for postdischarge interventions. Additionally, with the goal of avoiding long and burdensome assessments for patients and clinical staff, these results could be informative for those using the item bank to construct short forms.

Evaluation of the measurement properties of the Perceived Stress Scale (PSS) in Hispanic caregivers to patients with Alzheimer's disease and related disorders.

OBJECTIVES: The Perceived Stress Scale (PSS) is the most widely used measure of perceived stress; however, minimal psychometric evaluation has been performed among Hispanic respondents, and even less among Hispanic caregivers to persons with Alzheimer's disease and related disorders (ADRDs). DESIGN: Secondary data analysis. SETTING: New York City, NY, USA. PARTICIPANTS: A sample of 453 community dwelling Hispanic caregivers to patients with ADRD. MEASUREMENTS: Latent variable models were used to evaluate the PSS. Exploratory and confirmatory factor analyses were used to examine unidimensionality. Differential item functioning (DIF) was examined for age, education, and language using the graded item response model. RESULTS: The factor and bifactor analyses results supported essential unidimensionality of the item set; however, positively worded items were observed using response item theory to be less informative than the negatively worded items. Reliability estimates were high. Salient DIF was not observed for age, education, or language of interview using the primary DIF detection method. Sensitivity analyses using a second DIF detection method identified uniform language-DIF for the item, "In the last month, how often have you felt that you were on top of things?" However, the non-compensatory DIF value was below the threshold considered salient. CONCLUSIONS: In summary, the 10-item PSS performed well in a sample of English- and Spanish-speaking Hispanic caregivers to patients with ADRD. Very little DIF, and none of high magnitude and impact, was observed. However, the negatively worded items, perhaps because they are more directly reflective of stress, were more informative. In the context of a short-form measure or computerized adaptive test, more informative items are those that would be selected for inclusion.

Evaluation of measurement equivalence of the Family Satisfaction with the End-of-Life Care (FAMCARE): Tests of differential item functioning between Hispanic and non-Hispanic White caregivers.

OBJECTIVE: Although the psychometric properties of the Family Satisfaction with End-of-Life Care measure have been examined in diverse settings internationally; little evidence exists regarding measurement equivalence in Hispanic caregivers. The aim was to examine the psychometric properties of a short-form of the FAMCARE in Hispanics using latent variable models and place information on differential item functioning (DIF) in an existing family satisfaction item bank. METHOD: The graded form of the item response theory model was used for the analyses of DIF; sensitivity analyses were performed using a latent variable logistic regression approach. Exploratory and confirmatory factor analyses to examine dimensionality were performed within each subgroup studied. The sample included 1,834 respondents: 317 Hispanic and 1,517 non-Hispanic White caregivers of patients with Alzheimer's disease and cancer, respectively. RESULTS: There was strong support for essential unidimensionality for both Hispanic and non-Hispanic White subgroups. Modest DIF of low magnitude and impact was observed; flagged items related to information sharing. Only 1 item was flagged with significant DIF by both a primary and sensitivity method after correction for multiple comparisons: "The way the family is included in treatment and care decisions." This item was more discriminating for the non-Hispanic, White responders than for the Hispanic subsample, and was also a more severe indicator at some levels of the trait; the Hispanic respondents located at higher satisfaction levels were more likely than White non-Hispanic respondents to report satisfaction. SIGNIFICANCE OF RESULTS: The magnitude of DIF was below the salience threshold for all items. Evidence supported the measurement equivalence and use for cross-cultural comparisons of the short-form FAMCARE among Hispanic caregivers, including those interviewed in Spanish.

Evaluation of measurement equivalence of the Family Satisfaction with the End-of-Life Care in an ethnically diverse cohort: tests of differential item functioning.

BACKGROUND: The Family Satisfaction with End-of-Life Care is an internationally used measure of satisfaction with cancer care. However, the Family Satisfaction with End-of-Life Care has not been studied for equivalence of item endorsement across different socio-demographic groups using differential item functioning. AIMS: The aims of this secondary data analysis were (1) to examine potential differential item functioning in the family satisfaction item set with respect to type of caregiver, race, and patient age, gender, and education and (2) to provide parameters and documentation of differential item functioning for an item bank. DESIGN: A mixed qualitative and quantitative analysis was conducted. A priori hypotheses regarding potential group differences in item response were established. Item response theory and Wald tests were used for the analyses of differential item functioning, accompanied by magnitude and impact measures. RESULTS: Very little significant differential item functioning was observed for patient's age and gender. For race, 13 items showed differential item functioning after multiple comparison adjustment, 10 with non-uniform differential item functioning. No items evidenced differential item functioning of high magnitude, and the impact was negligible. For education, 5 items evidenced uniform differential item functioning after adjustment, none of high magnitude. Differential item functioning impact was trivial. One item evidenced differential item functioning for the caregiver relationship variable. CONCLUSION: Differential item functioning was observed primarily for race and education. No differential item functioning of high magnitude was observed for any item, and the overall impact of differential item functioning was negligible. One item, satisfaction with "the patient's pain relief," might be singled out for further study, given that this item was both hypothesized and observed to show differential item functioning for race and education.

Treatment of Patients Presenting With Suicidal Behavior in North American Pediatric Emergency Departments.

OBJECTIVE: This study sought to identify current practices for the treatment of patients presenting with suicidal ideation or a recent suicide attempt in pediatric emergency departments (EDs) in North America. METHODS: From October 10, 2018, to January 19, 2020, the authors conducted a cross-sectional online survey on current practices of pediatric emergency medicine chiefs practicing in the United States and Canada. RESULTS: Forty-six (34%) of 136 chiefs of pediatric emergency medicine responded to the survey. The three most frequent improvements chiefs reported they would like to see in the care of young patients with suicidal ideation or suicide attempt were easier access to mental health personnel for evaluations, having mental health personnel take primary responsibility for patient evaluation and treatment, and better access to mental health personnel for dispositional planning. CONCLUSIONS: The findings highlight the need for better mental health care in pediatric EDs to serve patients at increased risk for suicide.

Third-party callers to the national suicide prevention lifeline: Seeking assistance on behalf of people at imminent risk of suicide.

OBJECTIVE: To assist suicidal individuals, people in their social network are often directed to the National Suicide Prevention Lifeline (Lifeline). The study's objective was to provide information on third-party calls made out of concern for another person. METHOD: Reports on 172 third-party calls concerning individuals deemed to be at imminent suicide risk were completed by 30 crisis counselors at six Lifeline crisis centers. RESULTS: Third-party callers were most likely to be calling about a family member or friend and were significantly more likely than persons at risk to be female and middle-aged or older. Counselors were able to collect information about suicide risk, and counselors and third-parties were nearly always able to identify at least one intervention to aid the person at risk. Emergency services were contacted on 58.1 percent of the calls, which represents a somewhat higher rate of emergency services involvement than previously reported on imminent risk calls placed by the person at risk. Characteristics of third-parties and persons-at-risk each predicted emergency service involvement, but counselor characteristics did not. Non-emergency interventions were implemented on 68.6 percent of calls. CONCLUSIONS: Individuals calling the Lifeline when they are worried about someone are provided a range of interventions which can supplement, and at times replace, calling 911.

Lifeline Crisis Chat: Coding form development and findings on chatters' risk status and counselor behaviors.

OBJECTIVE: This study aimed to develop a reliable tool for the abstraction of data from crisis chat transcripts; to describe chatters' suicide risk status and selected counselor behaviors; and to examine the relationship of chatters' self-reported pre-chat suicidal thoughts to counselor behaviors and to chatters' disclosures of suicide risk during the chat conversation. METHODS: Coders used an instrument developed for this study to abstract data from 1034 crisis chats handled by the National Suicide Prevention Lifeline Crisis Chat network in 2015. The relationship of transcript coding data to data from an automated pre-chat survey (PCS) was examined. RESULTS: Lifeline Crisis Chat serves a young (median age = 21), high-risk population: 84.0% of chats (869/1034) came from chatters endorsing current or recent suicidal thoughts on the PCS. Counselors engaged in rapport-building on 93.3%, problem-solving on 70.1%, and suicide risk assessment on 67.7% of these 869 chats. Counselor risk assessment behavior, and the availability of information on suicide risk in the chat transcript, varied significantly by the chatter's PCS response. CONCLUSION: Crisis counselors are able to implement keystones of Lifeline's crisis intervention model over the medium of online chat. Additional efforts are needed to ensure that suicide risk is assessed on every chat.

Restrictive emotionality, depressive symptoms, and suicidal thoughts and behaviors among high school students.

Depression and suicidal thoughts and behaviors are prevalent among youth today. The current study sought to further our understanding of the correlates of depression and suicidality by assessing the relationship between restrictive emotionality (difficulty understanding and expressing emotions) and depressive symptoms and suicidal ideation and attempts among adolescents. A large group of high school students (n = 2189, 58.3% male; 13-18 years of age) completed a self-report survey as part of a 2-stage suicide screening project. Logistic regression analyses were used to assess the association between restrictive emotionality and depressive symptoms, suicidal ideation, and suicide attempts. Those reporting high restrictive emotionality were 11 times more likely to have elevated depressive symptom scores, 3 times more likely to report serious suicidal ideation (after controlling for depressive symptoms), and more than twice as likely to report a suicide attempt (after controlling for depressive symptoms) than those reporting low restrictive emotionality. Restrictive emotionality partially mediated the relationship between depressive symptoms and suicidal ideation and behavior. The pattern of association between restrictive emotionality and the outcome variables was similar for boys and girls. Restrictive emotionality is highly associated with elevated depressive symptoms and suicidal thoughts and behaviors among high school students, and may be a useful specific target in prevention and treatment efforts.

Differential Item Functioning Analyses of the Patient-Reported Outcomes Measurement Information System (PROMIS®) Measures: Methods, Challenges, Advances, and Future Directions.

Several methods used to examine differential item functioning (DIF) in Patient-Reported Outcomes Measurement Information System (PROMIS®) measures are presented, including effect size estimation. A summary of factors that may affect DIF detection and challenges encountered in PROMIS DIF analyses, e.g., anchor item selection, is provided. An issue in PROMIS was the potential for inadequately modeled multidimensionality to result in false DIF detection. Section 1 is a presentation of the unidimensional models used by most PROMIS investigators for DIF detection, as well as their multidimensional expansions. Section 2 is an illustration that builds on previous unidimensional analyses of depression and anxiety short-forms to examine DIF detection using a multidimensional item response theory (MIRT) model. The Item Response Theory-Log-likelihood Ratio Test (IRT-LRT) method was used for a real data illustration with gender as the grouping variable. The IRT-LRT DIF detection method is a flexible approach to handle group differences in trait distributions, known as impact in the DIF literature, and was studied with both real data and in simulations to compare the performance of the IRT-LRT method within the unidimensional IRT (UIRT) and MIRT contexts. Additionally, different effect size measures were compared for the data presented in Section 2. A finding from the real data illustration was that using the IRT-LRT method within a MIRT context resulted in more flagged items as compared to using the IRT-LRT method within a UIRT context. The simulations provided some evidence that while unidimensional and multidimensional approaches were similar in terms of Type I error rates, power for DIF detection was greater for the multidimensional approach. Effect size measures presented in Section 1 and applied in Section 2 varied in terms of estimation methods, choice of density function, methods of equating, and anchor item selection. Despite these differences, there was considerable consistency in results, especially for the items showing the largest values. Future work is needed to examine DIF detection in the context of polytomous, multidimensional data. PROMIS standards included incorporation of effect size measures in determining salient DIF. Integrated methods for examining effect size measures in the context of IRT-based DIF detection procedures are still in early stages of development.

National Suicide Prevention Lifeline crisis chat interventions: Evaluation of chatters' perceptions of effectiveness.

OBJECTIVE: As part of the National Suicide Prevention Lifeline's crisis response system, the Lifeline Crisis Chat Network (LCC) answers chats from hundreds of thousands of at-risk individuals yearly. The study's objective was to assess the effectiveness of these online crisis interventions. METHOD: Data from 39,911 pre-chat surveys and 13,130 linked pre- and post-chat surveys completed by LCC chatters from October 2017-June 2018 were analyzed. The relationship of several effectiveness measures with chatter demographics, pre-chat distress, suicidal ideation, and chatters' perceptions of engagement with their counselors was examined using a series of logistic regression analyses. RESULTS: Chatters were significantly and substantially less distressed at the end of the chat intervention than at the beginning. By the end of the chat, two-thirds of suicidal chatters reported that the chat had been helpful, while just under half reported being less suicidal. CONCLUSIONS: Our study offers empirical evidence for the Lifeline's online crisis chat services' effectiveness, but also highlights areas for improvement. This is of critical import in light of the recent designation of 988 as the nationwide number for the Lifeline beginning in 2022, which will increase the Lifeline's prominence in providing suicide prevention and mental health crisis interventions in the United States.

Assessing suicide risk among callers to crisis hotlines: a confirmatory factor analysis.

Our goal was to investigate the factor structure of a risk assessment tool utilized by suicide hotlines and to determine the predictive validity of the obtained factors in predicting subsequent suicidal behavior. We conducted an Exploratory Factor Analysis (EFA), an EFA in a Confirmatory Factor Analysis (EFA/CFA) framework, and a CFA on independent subsamples derived from a total sample of 1,085. Similar to previous studies, we found consistent evidence for a two-factor solution, with one factor representing a more pernicious form of suicide risk (i.e., Resolved Plans and Preparations; RPP) and one factor representing milder suicidal ideation (i.e., Suicidal Desire and Ideation; SDI). The RPP factor trended toward being more predictive of suicidal ideation at follow-up than the SDI factor.

The Relation between Descriptive Norms, Suicide Ideation, and Suicide Attempts among Adolescents.

This study examined the relationship between adolescents' beliefs about the prevalence of youth suicide ideation (ideation descriptive norms) and suicide attempts (attempt descriptive norms) with self-reported suicide ideation and attempts. Descriptive norms, suicide ideation, and suicide attempts as well as gender, race/ethnicity, and exposure to family, peer, and others' suicide were assessed in 2,109 students at six suburban New York State high schools. After controlling for demographic variables and exposure to suicide, elevated ideation descriptive norms and attempt descriptive norms were associated with higher rates of suicide ideation and lifetime suicide attempts among adolescents. Adolescents who believed suicide ideation and attempts to be more widespread among peers (i.e., elevated ideation and attempt descriptive norms) were more likely to endorse suicide ideation and attempts. Correcting these descriptive norms may be a worthwhile goal for school-based suicide prevention programs.

Peer victimization, depression, and suicidiality in adolescents.

The association between specific types of peer victimization with depression, suicidal ideation, and suicide attempts among adolescents was examined. A self-report survey was completed by 2,342 high-school students. Regression analyses indicated that frequent exposure to all types of peer victimization was related to high risk of depression, ideation, and attempts compared to students not victimized. Infrequent victimization was also related to increased risk, particularly among females. The more types of victimization the higher the risk for depression and suicidality among both genders. Specific types of peer victimization are a potential risk factor for adolescent depression and suicidality. It is important to assess depression and suicidality among victimized students in order to develop appropriate intervention methods.

Follow-up with Callers to the National Suicide Prevention Lifeline: Evaluation of Callers' Perceptions of Care.

Continuity of care for suicidal individuals engaged with a variety of health and mental health care systems has become a national priority, and crisis hotlines are increasingly playing a part in the risk management and continuum of care for these individuals. The current study evaluated a national initiative to have crisis centers in the National Suicide Prevention Lifeline network provide follow-up care to suicidal callers. Data were obtained from 550 callers followed by 41 crisis counselors from 6 centers. Two main data sources provided the information for the current study: a self-report counselor questionnaire on the follow-up activities completed on each clinical follow-up call and a telephone interview with follow-up clients, providing data on their perceptions of the follow-up intervention's effectiveness. The majority of interviewed follow-up clients reported that the intervention stopped them from killing themselves (79.6%) and kept them safe (90.6%). Counselor activities, such as discussing distractors, social contacts to call for help, and reasons for dying, and individual factors, such as baseline suicide risk, were associated with callers' perceptions of the impact of the intervention on their suicide risk. Our findings provide evidence that follow-up calls to suicidal individuals can reduce the perceived risk of future suicidal behavior.

Exposure to Suicide in High Schools: Impact on Serious Suicidal Ideation/Behavior, Depression, Maladaptive Coping Strategies, and Attitudes toward Help-Seeking.

Adolescents' exposure to a peer's suicide has been found to be associated with, as well as to predict, suicidal ideation and behavior. Although postvention efforts tend to be school-based, little is known about the impact of a schoolmate's suicide on the school's student population overall. The present study seeks to determine whether there is excess psychological morbidity among students in a school where a schoolmate has died by suicide, and whether students' attitudes about coping and help-seeking strategies are more or less problematic in such schools. Students in twelve high schools in Suffolk and Westchester counties in New York State-2865 students at six schools where a student had died by suicide within the past six months, and 2419 students at six schools where no suicide had occurred within the current students' tenure-completed an assessment of their suicidal ideation and behavior, depressive symptoms, coping and help-seeking attitudes, stressful life events, and friendship with suicide decedent (if applicable). No excess morbidity (i.e., serious suicidal ideation/behavior and depression) was evident among the general student population after a schoolmate's death by suicide; however, the risk of serious suicidal ideation/behavior was elevated among students at exposed schools who had concomitant negative life events. There was a significant relationship between friendship with the decedent and morbidity, in that students who were friends, but not close friends, of the decedents had the greatest odds of serious suicidal ideation/behavior. Overall, students in exposed schools had more adaptive attitudes toward help-seeking; but this was not true of the decedents' friends or students with concomitant negative life events. The implications of the findings for postvention strategies are discussed.

Bullying, depression, and suicidality in adolescents.

OBJECTIVE: To assess the association between bullying behavior and depression, suicidal ideation, and suicide attempts among adolescents. METHOD: A self-report survey was completed by 9th- through 12th-grade students (n = 2342) in six New York State high schools from 2002 through 2004. Regression analyses were conducted to examine the association between being victimized and bullying others with depression, ideation, and attempts. RESULTS: Approximately 9% of the sample reported being victimized frequently, and 13% reported bullying others frequently. Frequent exposure to victimization or bullying others was related to high risks of depression, ideation, and suicide attempts compared with adolescents not involved in bullying behavior. Infrequent involvement in bullying behavior also was related to increased risk of depression and suicidality, particularly among girls. The findings indicate that both victims and bullies are at high risk and that the most troubled adolescents are those who are both victims and bullies. Psychopathology was associated with bullying behavior both in and away from school. CONCLUSIONS: Victimization and bullying are potential risk factors for adolescent depression and suicidality. In evaluations of students involved in bullying behavior, it is important to assess depression and suicidality.

An evaluation of crisis hotline outcomes. Part 1: Nonsuicidal crisis callers.

The effectiveness of telephone crisis services/hotlines, examining proximal outcomes as measured by changes in callers' crisis state from the beginning to the end of their calls to eight centers in the U.S. and intermediate outcomes within 3 weeks of their calls, was evaluated. Between March 2003 and July 2004, 1,617 crisis callers were assessed during their calls and 801 (49.5%) participated in the followup assessment. Significant decreases in callers' crisis states and hopelessness were found during the course of the telephone session, with continuing decreases in crisis states and hopelessness in the following weeks. A majority of callers were provided with referrals and/or plans of actions for their concerns and approximately one third of those provided with mental health referrals had followed up with the referral by the time of the follow-up assessment. While crisis service staff coded these callers as nonsuicidal, at follow-up nearly 12% of them reported having suicidal thoughts either during or since their call to the center. The need to conduct suicide risk assessments with crisis callers and to identify strategies to improve referral follow-up is highlighted.

An evaluation of crisis hotline outcomes. Part 2: Suicidal callers.

In this study we evaluated the effectiveness of telephone crisis services/hotlines, examining proximal outcomes as measured by changes in callers' suicide state from the beginning to the end of their calls to eight centers in the U.S. and again within 3 weeks of their calls. Between March 2003 and July 2004, 1,085 suicide callers were assessed during their calls and 380 (35.0%) participated in the follow-up assessment. Several key findings emerged. Seriously suicidal individuals reached out to telephone crisis services. Significant decreases in suicidality were found during the course of the telephone session, with continuing decreases in hopelessness and psychological pain in the following weeks. A caller's intent to die at the end of the call was the most potent predictor of subsequent suicidality. The need to heighten outreach strategies and improve referrals is highlighted.

Suicide reporting content analysis: abstract development and reliability.

Despite substantial research on media influences and the development of media guidelines on suicide reporting, research on the specifics of media stories that facilitate suicide contagion has been limited. The goal of the present study was to develop a content analytic strategy to code features in media suicide reports presumed to be influential in suicide contagion and determine the interrater reliability of the qualitative characteristics abstracted from newspaper stories. A random subset of 151 articles from a database of 1851 newspaper suicide stories published during 1988 through 1996, which were collected as part of a national study in the United States to identify factors associated with the initiation of youth suicide clusters, were evaluated. Using a well-defined content-analysis procedure, the agreement between raters in scoring key concepts of suicide reports from the headline, the pictorial presentation, and the text were evaluated. The results show that while the majority of variables in the content analysis were very reliable, assessed using the kappa statistic, and obtained excellent percentages of agreement, the reliability of complicated constructs, such as sensationalizing, glorifying, or romanticizing the suicide, was comparatively low. The data emphasize that before effective guidelines and responsible suicide reporting can ensue, further explication of suicide story constructs is necessary to ensure the implementation and compliance of responsible reporting on behalf of the media.