AI enables phones to detect depression from facial cues, study shows
1 of 2 | A new smartphone application uses artificial intelligence to detect depression from facial cues, opening the door to real-time digital mental health support, a new research paper reports. Photo by Shutterbug75/Pixabay
A new smartphone application uses artificial intelligence to detect depression from facial cues, opening the door to real-time digital mental health support, a new research paper reports.
The researchers published their work Tuesday to the arXiv preprint database in advance of presenting it at the Association of Computing Machinery’s CHI 2024 conference in May.
Artificial intelligence coupled with facial-image processing software can reliably detect the onset of depression before the user knows something is wrong, according to the researchers in Dartmouth’s Department of Computer Science and Geisel School of Medicine in Hanover, N.H.
A prototype of a new smartphone application, MoodCapture uses a phone’s front camera to capture a person’s facial expressions and surroundings during regular use, and then evaluates the images for cues associated with depression.
“We undertook this study to address critical gaps in traditional methods of monitoring and detecting depression. Traditional approaches often involve self-reports and clinical assessments, which can be biased and may not capture the complexity of an individual’s mental state continuously,” the study’s co-first author, Subigya Nepal, a doctoral candidate in computer science, told UPI via email.
“MoodCapture aims to leverage the unguarded facial expressions captured during routine phone unlocks, envisioning a future where AI can assess mood in real time directly on the device, ensuring privacy and continuous mental health monitoring,” Nepal said.
He noted that this move toward more objective and unobtrusive methods holds promise for early detection of depression and timely intervention for at-risk individuals.
In a study of 177 people diagnosed with major depressive disorder, the app correctly identified early symptoms with 75% accuracy, the researchers said, noting that these results suggest that the technology could soon be publicly available.
“Over the next five years, we will see this technique used in clinical and everyday settings to help people at risk,” the study’s corresponding author, Andrew Campbell, the Albert Bradley 1915 Third Century Professor of Computer Science at Dartmouth, told UPI via email.
“A decade ago, we tried to see if images from the phone’s front-facing camera could be used to predict depression and failed,” Campbell said. “Today, the cameras on phones are orders of magnitude better, and new AI models allow us for the first time to accurately predict depression.”
Nepal added that “subtle, often overlooked cues can be meaningful indicators of mental health states.” This study “also shows that we may be able to make depression detection more accessible and less stigmatized by embedding it into the fabric of daily technology use without requiring explicit user input or clinical visits.”
The research is preliminary, so it’s necessary to interpret these results with caution, Dr. Gustavo Medeiros, a psychiatrist at the University of Maryland Medical Center in Baltimore, told UPI via email. He was not involved in the study.
The sample is relatively small, and the accuracy of prediction is suboptimal, and the diagnosis of depression was self-reported, which isn’t ideal, said Medeiros, who also is an assistant professor at the University of Maryland School of Medicine.
Even so, “this pilot study shows that if continued efforts are made, artificial intelligence may be used in psychiatry in a few years,” he said. “Scientific knowledge typically progresses inch by inch, and many additional steps are needed until this app can be reliably used.”
Most individuals with depression don’t seek help, so establishing a connection with a mental health clinician can be difficult for several reasons, Medeiros said.
“Although this app is unlikely to replace mental health professionals, it can help people by letting them know that they might be depressed, which may encourage them to seek psychiatric help” in early phases of the disorder, minimizing its negative effects, he said.
Medeiros added that “depression is a complex disease that affects the facial expression of patients in different ways.”
As a result, he suggests that future studies develop predictive models that go beyond facial expression — they should incorporate other types of passive data such as sleep and walking patterns, social media use and typing.
“Studying models with several sources of data will likely increase the accuracy of the prediction,” Medeiros said.
Biomarkers of depression, particularly those that are passively recorded and don’t require much effort from the user, are clearly needed, Dr. Dan V. Iosifescu, a professor of psychiatry and a member of the Neuroscience Institute at NYU Grossman School of Medicine, part of NYU Langone Health in New York City, told UPI via email.
“Current studies taking advantage of the ubiquity and sophistication of smartphones attempt to use elements of typing, voice, movement — accelerometers — and sleep as markers of depression severity,” Iosifescu said.
“Having a reliable tool to detect depression severity could help patients and clinicians by highlighting clinical need in undiagnosed cases, unexpected worsening of clinical course or emergent suicidal ideation,” he said.
While “human facial expressions are one of the best ways to detect emotions,” Iosifescu noted that the app’s 75% accuracy rate is modest, even though “the authors express confidence that future iterations might cross the threshold of 90% accuracy, at which point the application would indeed become clinically useful.”
He cautioned that “this research also has a potential dark side, as accurate and passive mood prediction could be potentially used for nefarious purposes by autocratic governments and by criminals searching for their next victim.”
Not all symptoms of depression — such as changes in sleep or appetite, difficulty concentrating, fatigue and feeling bad about oneself — can be evaluated via a photograph, Allison Kranich, a licensed clinical professional counselor at Northwestern Medicine McHenry Hospital in McHenry, Ill., told UPI via email.
“Perhaps facial recognition may be beneficial for mild and subclinical levels of depression, but I can’t see it significantly changing the course of a severe illness or saving lives of those experiencing suicidal ideations,” Kranich said.
She added that “we need more treating clinicians and better access to health care.”