Developing a Clinical Tool to Predict Suicide Risk

The following article was first published in Advances in Motion.  Advances in Motion provides health care professionals with information about the latest breakthroughs, research and clinical advances from Massachusetts General Hospital. You can subscribe to receive updates HERE.

Using Big Data to Identify Patients at Highest Risk for Suicide

Suicide is the second leading cause of death among young people aged 10 to 34, and 10th overall in the United States, with a steady rise over the past decade.

“Most people who attempt or die by suicide visited a healthcare provider in the months leading up to the event,” Dr. Smoller says. If people at risk for suicide are interacting with the healthcare system, then clinicians have real opportunities to identify them and intervene. However, he adds, “studies have shown that clinicians are not able to accurately predict who will attempt suicide.”

Clinicians are not very good at identifying who is really at risk. Unfortunately, we don’t do much better than chance in predicting whether somebody is going to make a suicide attempt or will die by suicide. That makes it very difficult to improve the prevention of suicide. If we had tools to better identify people in primary care settings who are at risk for a suicide attempt, we could focus interventions on those individuals.  Jordan Smoller, MD

Jordan Smoller, MD, SCD, a psychiatrist, epidemiologist and geneticist at Massachusetts General Hospital and colleagues at Mass General’s Center for Precision Psychiatry, along with collaborators at Boston Children’s Hospital and Harvard University decided to leverage large-scale, real-world data housed in the Mass General Brigham electronic health record and apply new machine learning techniques and artificial intelligence to develop a predictive model.

In 2017, they described a model based on data from 1.7 million patients in the American Journal of Psychiatry. The model incorporates both established risk factors such as substance abuse, mood disorders and psychiatric disorders, as well as somewhat less expected risk factors including certain injuries and chronic conditions such as infection and orthopedic conditions. The model was able to detect up to 45% of suicide attempts and deaths about two to three years in advance, with a specificity of 90%.

The team then decided to test the approach in patients at five additional healthcare systems in the United States. In 2020, they published an article in JAMA Network Open showing that the model performed just as well in these other settings.

They next conducted an extensive economic analysis and showed that their algorithm and others like it are cost-effective when combined with low-burden, evidence-based strategies for suicide prevention.

The team’s most recent research, published this year in JAMA Network Open, compared three methods for predicting risk of a suicide attempt:

• Clinician assessment
• A brief patient self-report scale
• The team’s predictive algorithm using data from the electronic health record to assign a risk score to each patient

The study included 1,818 patients who were seen in the Mass General psychiatric emergency department. At the one-month follow-up, 13% of patients contacted had attempted suicide. At the six-month follow-up, 22% of patients contacted had attempted suicide. Analysis showed that clinicians’ predictions of those suicide attempts were little better than chance. The best predictions were made by a model combining patient self-report and the electronic health record algorithm. Among those identified as high risk by this model, more than 30% had attempted suicide by one month, accounting for 65% of all suicide attempts. This study suggests that clinicians can significantly improve their ability to identify patients at high risk of suicide by using data from a brief patient self-report scale and health record data.

Suicide Risk Prediction Technology and Recommended Interventions

With this foundation of research, the team is now working to develop and disseminate a point-of-care screening tool housed in the electronic health record that provides decision support.

The goal is to target specific points in time when people are seen in primary care settings and may be at markedly increased risk for a suicide attempt:

• After discharge from psychiatric hospitalization
• After discharge from an acute care setting such as an emergency room

“Our goal is to build a seamless workflow that allows clinicians to identify relative risk levels and then guides them through an actual treatment plan or disposition plan that is appropriate to the patient’s level of risk,” Dr. Smoller says. “We’re making it relatively low friction and low burden for clinicians because they are incredibly busy and keeping track of many, many things.”

Because simply identifying individuals at increased risk for suicide is not enough—providers also need clinically actionable information—the tool will recommend specific actions based on risk level, including:

• Follow-up phone calls and texts
• Cognitive behavioral therapy designed specifically for patients at risk for suicide attempt
• Safety planning, which is a step-wise plan for patients to use when they experience suicidal thoughts or urges

The researchers hope to first develop a tool for use throughout the Mass General Brigham system and then to offer it nationwide.

During development, Dr. Smoller’s team conducted focus groups with clinicians from different hospital departments and consulted bioethicists to address issues involved in implementing such a tool. Dr. Smoller cautions, “This is a decision-support tool and is meant to inform rather than substitute for clinical judgment. In addition, the model is identifying clinical factors that best predict risk, but these factors are not necessarily causally related to suicide attempts. So, we can’t yet say that if you were to change a risk factor in the model, it would lower risk.”

Mass General’s Approach to Suicide Prevention

Mass General is uniquely poised to create a tool like this, Dr. Smoller says, due to a combination of excellent psychiatric care, expertise in data science and health informatics and world-class research in psychiatry. That multidisciplinary approach is at the heart of the recently formed Center for Precision Psychiatry.

“The goal of the center is to try to advance the field by bringing what we would call a precision medicine approach to psychiatry,” he says. “We’re leveraging many of the advances that have become possible in the past few years, with the advent of very large-scale data machine learning. We’re now able to bring together different streams of information with a real focus on implementation, actually bringing research to clinical care in ways that we couldn’t before.”

Barak-Corren Y, Castro VM, Javitt S, Hoffnagle AG, Dai Y, Perlis RH, Nock MK, Smoller JW, Reis BY. Predicting Suicidal Behavior From Longitudinal Electronic Health Records. Am J Psychiatry. 2017 Feb 1;174(2):154-162.

Barak-Corren Y, Castro VM, Nock MK, Mandl KD, Madsen EM, Seiger A, Adams WG, Applegate RJ, Bernstam EV, Klann JG, McCarthy EP, Murphy SN, Natter M, Ostasiewski B, Patibandla N, Rosenthal GE, Silva GS, Wei K, Weber GM, Weiler SR, Reis BY, Smoller JW. Validation of an Electronic Health Record-Based Suicide Risk Prediction Modeling Approach Across Multiple Health Care Systems. JAMA Netw Open. 2020 Mar 2;3(3):e201262.

Bentley KH, Zuromski KL, Fortgang RG, Madsen EM, Kessler D, Lee H, Nock MK, Reis BY, Castro VM, Smoller JW. Implementing Machine Learning Models for Suicide Risk Prediction in Clinical Practice: Focus Group Study With Hospital Providers. JMIR Form Res. 2022 Mar 11;6(3):e30946.

Ross EL, Zuromski KL, Reis BY, Nock MK, Kessler RC, Smoller JW. Accuracy Requirements for Cost-effective Suicide Risk Prediction Among Primary Care Patients in the US. JAMA Psychiatry. 2021 Jun 1;78(6):642-650.