Multi-Task Mental Health Detection with Large Language Models under Class Imbalance

Abstract

Social media posts provide a valuable opportunity for early mental health detection because they often capture users' thoughts, emotions, and distress signals in real time, potentially enabling timely identification of individuals at risk. However, building reliable models from such data is difficult, especially because benchmark datasets are often highly imbalanced across tasks and labels. In this paper, we study how loss design and reweighting strategies affect multi-task mental health prediction on four Reddit-based corpora covering six tasks: stress detection, binary depression detection, depression severity classification, suicidal ideation detection, binary suicide-risk detection, and five-level suicide-risk assessment. We compare several training configurations built on the same Qwen3-4B with LoRA backbone, including cross-entropy with reweighting, focal loss, focal loss with reweighting, a without-CE baseline, and a Proposed BACC-Guided setting that combines cross-entropy with a balanced-accuracy-oriented objective. Across the six tasks, the Proposed BACC-Guided setting achieves the best overall balanced accuracy, reaching a mean BACC of 0.62, compared with 0.59 for focal loss with reweighting, 0.58 for reweighting only, 0.57 for focal loss only, and 0.56 for the without-CE baseline. The gains are most evident on binary and moderately imbalanced tasks, suggesting that optimizing for balanced accuracy can produce more robust and generalizable mental health models under heterogeneous task imbalance. As an exploratory study, our findings provide initial insights that can guide future research on identifying the most suitable models and training strategies for multi-task mental health prediction.