Predicting risk of suicidal behavior from insurance claims data vs. linked data from insurance claims and electronic health records.

PURPOSE: Observational studies assessing effects of medical products on suicidal behavior often rely on health record data to account for pre-existing risk. We assess whether high-dimensional models predicting suicide risk using data derived from insurance claims and electronic health records (EHRs) are superior to models using data from insurance claims alone. METHODS: Data were from seven large health systems identified outpatient mental health visits by patients aged 11 or older between 1/1/2009 and 9/30/2017. Data for the 5 years prior to each visit identified potential predictors of suicidal behavior typically available from insurance claims (e.g., mental health diagnoses, procedure codes, medication dispensings) and additional potential predictors available from EHRs (self-reported race and ethnicity, responses to Patient Health Questionnaire or PHQ-9 depression questionnaires). Nonfatal self-harm events following each visit were identified from insurance claims data and fatal self-harm events were identified by linkage to state mortality records. Random forest models predicting nonfatal or fatal self-harm over 90 days following each visit were developed in a 70% random sample of visits and validated in a held-out sample of 30%. Performance of models using linked claims and EHR data was compared to models using claims data only. RESULTS: Among 15 845 047 encounters by 1 574 612 patients, 99 098 (0.6%) were followed by a self-harm event within 90 days. Overall classification performance did not differ between the best-fitting model using all data (area under the receiver operating curve or AUC = 0.846, 95% CI 0.839-0.854) and the best-fitting model limited to data available from insurance claims (AUC = 0.846, 95% CI 0.838-0.853). Competing models showed similar classification performance across a range of cut-points and similar calibration performance across a range of risk strata. Results were similar when the sample was limited to health systems and time periods where PHQ-9 depression questionnaires were recorded more frequently. CONCLUSION: Investigators using health record data to account for pre-existing risk in observational studies of suicidal behavior need not limit that research to databases including linked EHR data.

Variation in completeness of coding external cause of injuries under ICD-10-CM.

INTRODUCTION: Information about causes of injury is key for injury prevention efforts. Historically, cause-of-injury coding in clinical practice has been incomplete due to the need for extra diagnosis codes in the International Classification of Diseases-Ninth Revision-Clinical Modification (ICD-9-CM) coding. The transition to ICD-10-CM and increased use of clinical support software for diagnosis coding is expected to improve completeness of cause-of-injury coding. This paper assesses the recording of external cause-of-injury codes specifically for those diagnoses where an additional code is still required. METHODS: We used electronic health record and claims data from 10 health systems from October 2015 to December 2021 to identify all inpatient and emergency encounters with a primary diagnosis of injury. The proportion of encounters that also included a valid external cause-of-injury code is presented. RESULTS: Most health systems had high rates of cause-of-injury coding: over 85% in emergency departments and over 75% in inpatient encounters with primary injury diagnoses. However, several sites had lower rates in both settings. State mandates were associated with consistently high external cause recording. CONCLUSIONS: Completeness of cause-of-injury coding improved since the adoption of ICD-10-CM coding and increased slightly over the study period at most sites. However, significant variation remained, and completeness of cause-of-injury coding in any diagnosis data used for injury prevention planning should be empirically determined.

Management of Depression in Adults: A Review.

Importance: Approximately 9% of US adults experience major depression each year, with a lifetime prevalence of approximately 17% for men and 30% for women. Observations: Major depression is defined by depressed mood, loss of interest in activities, and associated psychological and somatic symptoms lasting at least 2 weeks. Evaluation should include structured assessment of severity as well as risk of self-harm, suspected bipolar disorder, psychotic symptoms, substance use, and co-occurring anxiety disorder. First-line treatments include specific psychotherapies and antidepressant medications. A network meta-analysis of randomized clinical trials reported cognitive therapy, behavioral activation, problem-solving therapy, interpersonal therapy, brief psychodynamic therapy, and mindfulness-based psychotherapy all had at least medium-sized effects in symptom improvement over usual care without psychotherapy (standardized mean difference [SMD] ranging from 0.50 [95% CI, 0.20-0.81] to 0.73 [95% CI, 0.52-0.95]). A network meta-analysis of randomized clinical trials reported 21 antidepressant medications all had small- to medium-sized effects in symptom improvement over placebo (SMD ranging from 0.23 [95% CI, 0.19-0.28] for fluoxetine to 0.48 [95% CI, 0.41-0.55] for amitriptyline). Psychotherapy combined with antidepressant medication may be preferred, especially for more severe or chronic depression. A network meta-analysis of randomized clinical trials reported greater symptom improvement with combined treatment than with psychotherapy alone (SMD, 0.30 [95% CI, 0.14-0.45]) or medication alone (SMD, 0.33 [95% CI, 0.20-0.47]). When initial antidepressant medication is not effective, second-line medication treatment includes changing antidepressant medication, adding a second antidepressant, or augmenting with a nonantidepressant medication, which have approximately equal likelihood of success based on a network meta-analysis. Collaborative care programs, including systematic follow-up and outcome assessment, improve treatment effectiveness, with 1 meta-analysis reporting significantly greater symptom improvement compared with usual care (SMD, 0.42 [95% CI, 0.23-0.61]). Conclusions and Relevance: Effective first-line depression treatments include specific forms of psychotherapy and more than 20 antidepressant medications. Close monitoring significantly improves the likelihood of treatment success.

Depression Treatment Initiation Among Patients With Versus Without Chronic Pain.

OBJECTIVE: The purpose of this study was to examine the extent to which the presence of chronic noncancer pain (CNCP) impacts the likelihood that patients with diagnoses of depression will initiate depression treatment compared with those without CNCP. METHODS: We performed a retrospective cohort study of Kaiser Permanente of Georgia members older than 18 years who received a diagnosis of depression. Demographics and medical history were extracted from the electronic health record database. Members were further classified by the presence or absence of a CNCP diagnosis. Outcomes of interest were treated as time dependent and included ( 1 ) time to fulfillment of a new antidepressant medication and ( 2 ) time to a follow-up mental health encounter. Outcomes were compared between members with and without a CNCP diagnosis using Kaplan-Meier survival curves and Cox proportional hazard regression models. RESULTS: During the study period, 22,996 members met the inclusion criteria and 27.4% had a diagnosis of CNCP. In the matched sample, there was no difference in the time to a new antidepressant fill among members with and without CNCP (hazard ratio = 0.96; 95% confidence interval = 0.90-1.02; p = .18). In contrast, members with CNCP were significantly less likely to have a new mental health encounter after diagnosis (hazard ratio = 0.87; 95% confidence interval = 0.81-0.94; p < .001). CONCLUSIONS: Patients with CNCP were significantly less likely to have a new mental health encounter after a depression diagnosis compared with patients without CNCP. Additional outreach and consideration may be needed to improve initiation of depression treatment for newly diagnosed patients with comorbid depression and CNCP.

Association Between Adverse Social Determinants of Health and Suicide Death.

OBJECTIVE: The aim of this study was to identify adverse social determinants of health (SDoH) International Statistical Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code prevalence among individuals who died by suicide and to examine associations between documented adverse SDoH and suicide. RESEARCH DESIGN: A case-control study using linked medical record, insurance claim, and mortality data from 2000 to 2015 obtained from 9 Mental Health Research Network-affiliated health systems. We included 3330 individuals who died by suicide and 333,000 randomly selected controls matched on index year and health system location. All individuals in the study (cases and controls) had at least 10 months of enrollment before the study index date. The index date for the study for each case and their matched controls was the suicide date for that given case. RESULTS: Adverse SDoH documentation was low; only 6.6% of cases had ≥1 documented adverse SDoH in the year before suicide. Any documented SDoH and several specific adverse SDoH categories were more frequent among cases than controls. Any documented adverse SDoH was associated with higher suicide odds [adjusted odds ratio (aOR)=2.76; 95% CI: 2.38-3.20], as was family alcoholism/drug addiction (aOR=18.23; 95% CI: 8.54-38.92), being an abuse victim/perpetrator (aOR=2.53; 95% CI: 1.99-3.21), other primary support group problems (aOR=1.91; 95% CI: 1.32-2.75), employment/occupational maladjustment problems (aOR=8.83; 95% CI: 5.62-13.87), housing/economic problems (aOR: 6.41; 95% CI: 4.47-9.19), legal problems (aOR=27.30; 95% CI: 12.35-60.33), and other psychosocial problems (aOR=2.58; 95% CI: 1.98-3.36). CONCLUSIONS: Although documented SDoH prevalence was low, several adverse SDoH were associated with increased suicide odds, supporting calls to increase SDoH documentation in medical records. This will improve understanding of SDoH prevalence and assist in identification and intervention among individuals at high suicide risk.

Factors associated with suicide mortality among reproductive age women: a case-control study.

OBJECTIVE: Examine demographic, psychosocial, pregnancy-related, and healthcare utilisation factors associated with suicide mortality among reproductive age women. METHODS: Data from nine health care systems in the Mental Health Research Network were included. A case-control study design was used in which 290 reproductive age women who died by suicide (cases) from 2000 to 2015 were matched with 2,900 reproductive age women from the same healthcare system who did not die by suicide (controls). Conditional logistic regression was used to analyse associations between patient characteristics and suicide. RESULTS: Women of reproductive age who died by suicide were more likely to have mental health (aOR = 7.08, 95% CI: 5.17, 9.71) or substance use disorders (aOR = 3.16, 95% CI: 2.19, 4.56) and to have visited the emergency department in the year prior to index date (aOR = 3.47, 95% CI: 2.50, 4.80). Non-Hispanic White women (aOR = 0.70, 95% CI: 0.51, 0.97) and perinatal (pregnant or postpartum) women were less likely to have died by suicide (aOR = 0.27, 95% CI: 0.13, 0.58). CONCLUSIONS: Reproductive age women with mental health and/or substance use disorders, prior emergency department encounters, or who are of racial or ethnic minority status were at increased risk of suicide mortality and may benefit from routine screening and monitoring. Future research should further examine the relationship between pregnancy-related factors and suicide mortality.

Effect of Offering Care Management or Online Dialectical Behavior Therapy Skills Training vs Usual Care on Self-harm Among Adult Outpatients With Suicidal Ideation: A Randomized Clinical Trial.

Importance: People at risk of self-harm or suicidal behavior can be accurately identified, but effective prevention will require effective scalable interventions. Objective: To compare 2 low-intensity outreach programs with usual care for prevention of suicidal behavior among outpatients who report recent frequent suicidal thoughts. Design, Setting, and Participants: Pragmatic randomized clinical trial including outpatients reporting frequent suicidal thoughts identified using routine Patient Health Questionnaire depression screening at 4 US integrated health systems. A total of 18 882 patients were randomized between March 2015 and September 2018, and ascertainment of outcomes continued through March 2020. Interventions: Patients were randomized to a care management intervention (n = 6230) that included systematic outreach and care, a skills training intervention (n = 6227) that introduced 4 dialectical behavior therapy skills (mindfulness, mindfulness of current emotion, opposite action, and paced breathing), or usual care (n = 6187). Interventions, lasting up to 12 months, were delivered primarily through electronic health record online messaging and were intended to supplement ongoing mental health care. Main Outcomes and Measures: The primary outcome was time to first nonfatal or fatal self-harm. Nonfatal self-harm was ascertained from health system records, and fatal self-harm was ascertained from state mortality data. Secondary outcomes included more severe self-harm (leading to death or hospitalization) and a broader definition of self-harm (selected injuries and poisonings not originally coded as self-harm). Results: A total of 18 644 patients (9009 [48%] aged 45 years or older; 12 543 [67%] female; 9222 [50%] from mental health specialty clinics and the remainder from primary care) contributed at least 1 day of follow-up data and were included in analyses. Thirty-one percent of participants offered care management and 39% offered skills training actively engaged in intervention programs. A total of 540 participants had a self-harm event (including 45 deaths attributed to self-harm and 495 nonfatal self-harm events) over 18 months following randomization: 172 (3.27%) in care management, 206 (3.92%) in skills training, and 162 (3.27%) in usual care. Risk of fatal or nonfatal self-harm over 18 months did not differ significantly between the care management and usual care groups (hazard ratio [HR], 1.07; 97.5% CI, 0.84-1.37) but was significantly higher in the skills training group than in usual care (HR, 1.29; 97.5% CI, 1.02-1.64). For severe self-harm, care management vs usual care had an HR of 1.03 (97.5% CI, 0.71-1.51); skills training vs usual care had an HR of 1.34 (97.5% CI, 0.94-1.91). For the broader self-harm definition, care management vs usual care had an HR of 1.10 (97.5% CI, 0.92-1.33); skills training vs usual care had an HR of 1.17 (97.5% CI, 0.97-1.41). Conclusions and Relevance: Among adult outpatients with frequent suicidal ideation, offering care management did not significantly reduce risk of self-harm, and offering brief dialectical behavior therapy skills training significantly increased risk of self-harm, compared with usual care. These findings do not support implementation of the programs tested in this study. Trial Registration: ClinicalTrials.gov Identifier: NCT02326883.

Harmonized Outcome Measures for Use in Depression Patient Registries and Clinical Practice.

Major depressive disorder is a common mental health condition that affects an estimated 16.2 million adults and 3.1 million adolescents in the United States. Yet, a lack of uniformity remains in measurements and monitoring for depression both in clinical practice and in research settings. This project aimed to develop a minimum set of standardized outcome measures relevant to both patients and clinicians that can be collected in depression registries and clinical practice. Twenty-nine depression registries and related data collection efforts were identified and invited to submit outcome measures. Additional measures were identified through literature searches and reviews of quality measures. A multistakeholder panel representing clinicians; payers; government agencies; industry; and medical specialty, health care quality, and patient advocacy organizations categorized the 27 identified measures using the Agency for Healthcare Research and Quality's supported Outcome Measures Framework. The panel identified 10 broadly relevant measures and harmonized definitions for these measures through in-person and virtual meetings. The harmonized measures represent a minimum set of outcomes that are relevant to clinicians and patients and appropriate for use in depression research and clinical practice. Routine and consistent collection of these measures in registries and other systems would support creation of a national research infrastructure to efficiently address new questions, improve patient management and outcomes, and facilitate care coordination.

Clinical risk prediction models and informative cluster size: Assessing the performance of a suicide risk prediction algorithm.

Clinical visit data are clustered within people, which complicates prediction modeling. Cluster size is often informative because people receiving more care are less healthy and at higher risk of poor outcomes. We used data from seven health systems on 1,518,968 outpatient mental health visits from January 1, 2012 to June 30, 2015 to predict suicide attempt within 90 days. We evaluated true performance of prediction models using a prospective validation set of 4,286,495 visits from October 1, 2015 to September 30, 2017. We examined dividing clustered data on the person or visit level for model training and cross-validation and considered a within cluster resampling approach for model estimation. We evaluated optimism by comparing estimated performance from a left-out testing dataset to performance in the prospective dataset. We used two prediction methods, logistic regression with least absolute shrinkage and selection operator (LASSO) and random forest. The random forest model using a visit-level split for model training and testing was optimistic; it overestimated discrimination (area under the curve, AUC = 0.95 in testing versus 0.84 in prospective validation) and classification accuracy (sensitivity = 0.48 in testing versus 0.19 in prospective validation, 95th percentile cut-off). Logistic regression and random forest models using a person-level split performed well, accurately estimating prospective discrimination and classification: estimated AUCs ranged from 0.85 to 0.87 in testing versus 0.85 in prospective validation, and sensitivity ranged from 0.15 to 0.20 in testing versus 0.17 to 0.19 in prospective validation. Within cluster resampling did not improve performance. We recommend dividing clustered data on the person level, rather than visit level, to ensure strong performance in prospective use and accurate estimation of future performance at the time of model development.

Should we hide the eyes of the dead? Continuing bonds in Kanaké Paicî-Camuki Country.

This paper describes and analyzes the ethnopsychology and ethnopsychiatry of the Kanaké of the linguistic Paicî-Camuki country, of New Caledonia. The Kanaké advocate for the forfeiting of past aspects of the relationship with deceased loved ones, and the creation of new ones with them as an ancestor. Whenever the past bond remains too strong, putting the relatives at risk by the ancestors' sustained presence, Kanaké healers and initiates may act upon the invisible world to preserve the living by severing the bond. Through the case-study of Sunia, I examine how the bereaved navigate between these norms and resources to develop unique coping solutions.

Using predictive analytics to improve pragmatic trial design.

Clinical trials embedded in health systems can randomize large populations using automated data sources to determine trial eligibility and assess outcomes. The suicide prevention outreach trial used real-world data for trial design and randomized 18,868 individuals in four health systems using patient-reported thoughts of death or self-harm (Patient Health Questionnaire item 9). This took 3.5 years. We consider if using predictive analytics, that is, suicide risk estimates based on prediction models, could improve trial "efficiency." We used data on mental health outpatient visits between 1 January 2009 and 30 September 2017 in seven health systems (HealthPartners; Henry Ford Health System; and Colorado, Hawaii, Northwest, Southern California, and Washington Kaiser Permanente regions). We used a suicide risk prediction model developed in these same systems. We compared five trial designs with different eligibility criteria: a response of a 2 or 3 on Patient Health Questionnaire item 9, a response of a 3, suicide risk score above 90th, 95th, or 99th percentile. We compared the sample that met each criterion, 90-day suicide attempt rate following first eligible visit, and necessary sample sizes to detect a 15%, 25%, and 35% relative reduction in the suicide attempt rate, assuming 90% power, for each eligibility criterion. Our sample included 24,355,599 outpatient visits. Despite wide-spread use of Patient Health Questionnaire, 21,026,985 (86.3%) visits did not have a recorded Patient Health Questionnaire. Of the 2,928,927 individuals in our sample, 109,861 had a recorded Patient Health Questionnaire item 9 response of a 2 or 3 over the study years with a 1.40% 90-day suicide attempt rate and 50,047 had a response of a 3 (suicide attempt rate 1.98%). More patients met criteria requiring a certain risk score or higher: 331,273 had a 90th percentile risk score or higher (suicide attempt rate: 1.36%); 182,316 a 95th percentile or higher (suicide attempt rate 2.16%), and 78,655 a 99th percentile or higher (suicide attempt rate: 3.95%). Eligibility criterion of a Patient Health Questionnaire item 9 response of a 2 or 3 would require randomizing 44,081 individuals (40.2% of eligible population in our sample); eligibility criterion of a 3 would require 31,024 individuals (62.0% of eligible population). Eligibility criterion of a suicide risk score of 90th percentile or higher would require 45,675 individuals (13.8% of eligible population), 95th percentile 28,699 individuals (15.7% of eligible population), and 99th percentile 15,509 (19.7% of eligible population). A suicide risk prediction calculator could improve trial "efficiency"; identifying more individuals at increased suicide risk than relying on patient-report. It is an open scientific question if individuals identified using predictive analytics would respond differently to interventions than those identified by more traditional means.

Effectiveness of clinical decision support systems and telemedicine on outcomes of depression: a cluster randomized trial in general practice.

BACKGROUND: Computerized Clinical Decision Support Systems (CCDSS) are information technology tools, designed to improve clinical decision-making. Telemedicine is a health care service delivery using videoconferencing, telephone or messaging technologies. OBJECTIVES: Our project aimed at testing the effectiveness of a composite CCDSS and telemedicine approach designed to treat depression in primary care. METHODS: This cluster randomized trial involved four GP clinics located in Northern Italy. Two clinics were assigned to the experimental protocol, and two served as controls. The study compared the telemedicine group (TG), in which GPs had access to a CCDSS platform, with the control group (CG) in which GPs provided treatment as usual (TAU). Patients scoring ≥11 on Patient Heath Questionnaire and ≥26 on the Inventory of Depressive Symptomatology-Self-Report were eligible for participation. Patients were also administered the World Health Organization Quality of Life-BREF to assess quality of life and Medical Interview Satisfaction Scale 21 to assess satisfaction with the medical interview. RESULTS: Overall, 2810 patients were screened and 66 in the experimental group and 32 in the CG passed the screening stages and met inclusion criteria. The percentage of remitters at 6 months was significantly higher in the TG than in the CG group (24.1% versus 3.1%, χ 2 = 6.6, P = 0.01). This difference remained significant after adjusting for baseline confounders. Physical and psychological quality of life improved significantly from baseline in both groups. Patients reported, on average, good satisfaction with the medical interview. CONCLUSIONS: Our study showed that a combined CCDSS and telemedicine approach may be more effective than the TAU offered by GPs to patients with depression. TRIAL REGISTRATION: The trial was registered on https://clinicaltrials.gov/ on 5 October 2012 with identifier: NCT01701791. The first participant was enrolled on 5 May 2014 and the study was completed on May 2016.

Variation in patterns of health care before suicide: A population case-control study.

BACKGROUND: The United States has experienced a significant rise in suicide. As decision makers identify how to address this national concern, healthcare systems have been identified as an optimal location for prevention. OBJECTIVE: To compare variation in patterns of healthcare use, by health setting, between individuals who died by suicide and the general population. DESIGN: Case-Control Study. SETTING: Eight healthcare systems across the United States. PARTICIPANTS: 2674 individuals who died by suicide between 2000 and 2013 along with 267,400 individuals matched on time-period of health plan membership and health system affiliation. MEASUREMENTS: Healthcare use in the emergency room, inpatient hospital, primary care, and outpatient specialty setting measured using electronic health record data during the 7-, 30-, 60-, 90-, 180-, and 365-day time periods before suicide and matched index date for controls. RESULTS: Healthcare use was more common across all healthcare settings for individuals who died by suicide. Nearly 30% of individuals had a healthcare visit in the 7-days before suicide (6.5% emergency, 16.3% outpatient specialty, and 9.5% primary care), over half within 30 days, and >90% within 365 days. Those who died by suicide averaged 16.7 healthcare visits during the year. The relative risk of suicide was greatest for individuals who received care in the inpatient setting (aOR = 6.23). There was both a large relative risk (aOR = 3.08) and absolute utilization rate (43.8%) in the emergency room before suicide. LIMITATIONS: Participant race/ethnicity was not available. The sample did not include uninsured individuals. CONCLUSIONS: This study provides important data about how care utilization differs for those who die by suicide compared to the general population and can inform decision makers on targeting of suicide prevention activities within health systems.

Improving pragmatic clinical trial design using real-world data.

BACKGROUND: Pragmatic clinical trials often use automated data sources such as electronic health records, claims, or registries to identify eligible individuals and collect outcome information. A specific advantage that this automated data collection often yields is having data on potential participants when design decisions are being made. We outline how this data can be used to inform trial design. METHODS: Our work is motivated by a pragmatic clinical trial evaluating the impact of suicide-prevention outreach interventions on fatal and non-fatal suicide attempts in the 18 months after randomization. We illustrate our recommended approaches for designing pragmatic clinical trials using historical data from the health systems participating in this study. Specifically, we illustrate how electronic health record data can be used to inform the selection of trial eligibility requirements, to estimate the distribution of participant characteristics over the course of the trial, and to conduct power and sample size calculations. RESULTS: Data from 122,873 people with patient health questionnaire (PHQ) responses, recorded in their electronic health records between 1 July 2010 and 31 March 2012, were used to show that the suicide attempt rate in the 18 months following completion of the questionnaire varies by response to item nine of the PHQ. We estimated that the proportion of individuals with a prior recorded elevated PHQ (i.e. history of suicidal ideation) would decrease from approximately 50% at the beginning of a trial to about 5%, 50 weeks later. Using electronic health record data, we conducted simulations to estimate the power to detect a 25% reduction in suicide attempts. Simulation-based power calculations estimated that randomizing 8000 participants per randomization arm would allow 90% power to detect a 25% reduction in the suicide attempt rate in the intervention arm compared to usual care at an alpha rate of 0.05. CONCLUSIONS: Historical data can be used to inform the design of pragmatic clinical trials, a strength of trials that use automated data collection for randomizing participants and assessing outcomes. In particular, realistic sample size calculations can be conducted using real-world data from the health systems in which the trial will be conducted. Data-informed trial design should yield more realistic estimates of statistical power and maximize efficiency of trial recruitment.

The Mediating Effect of Sleep Disturbance on the Relationship Between Nonmalignant Chronic Pain and Suicide Death.

IMPORTANCE: Few studies have examined the relationship between nonmalignant chronic pain (NMCP) and suicide death, and even fewer have specifically explored what role sleep disturbance might play in the association between NMCP and suicide death. OBJECTIVE: To assess whether sleep disturbance mediates the relationship between NMCP and suicide death. DESIGN: This case-control study included 2,674 individuals who died by suicide between 2000 and 2013 (cases) and 267,400 matched individuals (controls). SETTING: Eight Mental Health Research Network (MHRN)-affiliated healthcare systems. PARTICIPANTS: All cases and matched controls were health plan members for at least 10 months during the year prior to the index date. MAIN OUTCOMES AND MEASURES: Sociodemographic data and diagnosis codes for NMCP and sleep disorders were extracted from the MHRN's Virtual Data Warehouse. Suicide mortality was identified using International Statistical Classification of Diseases and Related Health Problems (ICD)-10 codes from official government mortality records matched to health system records. RESULTS: After accounting for covariates, there was a significant relationship between NMCP and sleep disturbance; those who were diagnosed with NMCP were more likely to develop subsequent sleep disturbance. Similarly, sleep disturbance was significantly associated with suicide death. Finally, a significant indirect effect of NMCP on suicide death, through sleep disturbance, and a nonsignificant direct effect of NMCP on suicide death provide support for a fully mediated model. CONCLUSIONS AND RELEVANCE: There is a need for clinicians to screen for both sleep disturbance and suicidal ideation in NMCP patients and for health systems to implement more widespread behavioral treatments that address comorbid sleep problems and NMCP.

Counter-Point: Early Warning Systems Are Imperfect, but Essential.

Sequential analysis can be used as an early warning system about potential unintended consequences of health policy decisions, generating follow-up investigations, but it should not be used as causal evidence.

Counter-Point: Staying Honest When Policy Changes Backfire.

Despite the good intentions of the Food and Drug Administration (FDA), many drug warnings are ineffective or have unintended consequences, particularly if the media exaggerates the messages and scares the public. The controversial 2003 to 2004 FDA warnings on youth suicidality associated with antidepressant use are a case in point. In a 10-year interrupted time series (ITS) analysis in 11 health plans, we found that the warnings and hyped media coverage led to substantial reductions in antidepressant use (declines in antidepressant use and overall care corroborated in several studies), and small, visible increases in emergency room and inpatient poisonings with psychotropic drugs. In a gross misunderstanding of the method, Dr Stone calls ITS, "an intuition based upon false analogies, fallacious assumptions and analytical error." We demonstrate visually using published studies that the ITS method is one of the oldest (hundreds of years) and strongest quasi-experimental study designs, and that the alternative data analyses proposed by Dr Stone do not have rates (denominators), nor baselines, so the measures of change are invalid.