Towards Explainable and Safe Systems for Health Data

Abstract

The integration of artificial intelligence into healthcare holds immense potential to transform medical services through personalized care, early diagnosis, and more informed decision-making. However, despite significant advances in digital health technologies, the adoption of AI-driven systems remains limited, particularly in sensitive domains such as mental health, due to concerns around transparency, safety, and user trust. These concerns are compounded by the complexity of clinical language, the high stakes of decision-making, and the need for accountability in automated systems. This research addresses these challenges by investigating how Large Language Models (LLMs) and related techniques can be adapted to create explainable, interpretable, and safe health information systems. Positioned at the intersection of health informatics, Natural Language Processing (NLP), and Information Retrieval (IR), this work explores both the potential and limitations of LLMs in clinical contexts. While LLMs are effective at processing large textual datasets and producing fluent responses, their black-box nature, tendency to hallucinate, and associated privacy risks make them unsuitable for direct application in healthcare without significant adaptation. This thesis proposes methodologies and systems focused on improving explanation generation, clinical relevance, and the safety of model outputs in health-related queries. The research is guided by eight specific goals: (1) Characterizing health-related language: Modeling the vocabulary used by individuals with specific health conditions using relevance-based statistical language models; (2) Generating natural language explanations: Designing interpretable models that explain classification outputs in ways meaningful to clinicians; (3) Adapting LLMs to clinical reasoning: Fine-tuning LLMs to reflect clinical workflows, particularly for retrieving symptom evidence and supporting diagnostic decisions; (4) Improving answer retrieval: Developing search systems for health queries with better result quality and contextual relevance; (5) Building health recommendation systems: Creating personalized tools for discovering reliable and relevant health information; (6) Ensuring safety and reliability: Creating ''guardrails'' to reduce hallucinations, biases, and misinformation in model outputs; and (7) Supporting multilingual and under-resourced settings: Extending solutions to diverse linguistic and cultural environments. Substantial progress has already been made. Goals G2 and G3 led to a Q1 journal publication on explainable depression detection using social media data, where we proposed and evaluated two architectures: a dual-model approach separating classification and explanation, and a unified LLM-based model that performs both tasks. For G4, we developed MindWell, a conversational agent for depression screening. Based on transformer models and grounded in the Beck Depression Inventory-II (BDI-II), MindWell contextualizes user-generated content from social media to assist clinicians in identifying emotional patterns and symptoms. Ongoing work on G1 involves using statistical language models to identify lexical patterns linked to depressive behaviors online. For G7, we have trained an LLM adapted to Galician, an underrepresented co-official language in Spain, and are currently evaluating its performance in localized health information contexts.