Bias Beneath the Tone: Empirical Characterisation of Tone Bias in LLM-Driven UX Systems

Large Language Models are increasingly used in conversational systems such as digital Personal Assistants, shaping how people interact with technology through language. While their responses often sound fluent and natural, they can also carry subtle tone biases such as sounding overly polite, cheerful, or cautious even when neutrality is expected. These tendencies can influence how users perceive trust, empathy, and fairness in dialogue. In this study, we explore tone bias as a hidden behavioural trait of LLMs. The novelty of this research lies in the integration of controllable LLM-based dialogue synthesis with tone classification models, enabling robust and ethical tone detection in PA interactions. We created two synthetic dialogue datasets: one generated from neutral prompts and another explicitly guided to produce positive or negative tones. Surprisingly, even the neutral set showed consistent tonal skew, suggesting that bias may stem from the model’s underlying conversational style. Using weak supervision through a pretrained DistilBERT model, we labelled tones and trained several classifiers to detect these patterns. Ensemble models achieved macro-F1 scores up to 0.92, showing that tone bias is systematic, measurable, and relevant to designing fair and trustworthy conversational AI.

Paper

Similar papers

© 2026 NYSGPT2525 LLC