Advances in generative artificial intelligence (GenAI) have prompted interest in its application in sensitive fields, including mental health. Yet the validity of its nonverbal social-cognitive abilities remains unclear. This study addresses two issues that probe GenAI’s social intelligence: its capacity to interpret nonverbal channels (bodily gestures and vocal tone) with accuracy comparable to humans, and whether accuracy of emotion recognition is different by emotional valence.
Emotion recognition accuracy of three Gemini GenAI models (Pro 1.5, Pro 2 (Experimental) and Flash 2) were evaluated using the EU-Emotion Stimulus Set of bodily gestures and vocal tone depicting basic emotions. Model accuracy was compared to validated human benchmarks. Advanced GenAI models achieved near-human accuracy in recognizing emotions from vocal tone.
However, their performance was significantly lower than that of humans in interpreting bodily gestures. GenAI models recognized positive emotions more accurately than negative ones. This bias was significant across all models for bodily gestures and in the advanced models for vocal tone.
GenAI models exhibit a non-humanlike social-cognitive profile, excelling with positive emotions but struggling with the negative content. This finding carries profound clinical and theoretical relevance, highlighting the risks of integrating GenAI into sensitive fields and underscoring the need for continued validation. The advent and widespread adoption of Generative Artificial Intelligence (GenAI) technologies, driven by the rapid evolution of Large Language Models (LLMs), have ignited substantial public interest, demonstrating remarkable capabilities across a vast array of applications, including programming, legal analysis, education, healthcare, and psychology1,2,3.
Since the public release of ChatGPT, reports indicate that hundreds of millions of users are now leveraging AI4, with many seeking informal mental health support5,6,7. While these initial applications relied heavily on text-based LLMs, the current study expands this focus by evaluating advanced, natively multimodal GenAI systems. This burgeoning reliance on GenAI, particularly in sensitive domains, necessitates dedicated research into the capabilities of these models8,9, specifically concerning their proficiency in emotion recognition.
Social cognition, encompassing the processes by which individuals perceive, interpret, and respond to social information, serves as a foundational component of adaptive social functioning10. A core component of social cognition is mentalization, the capacity to understand mental states in oneself and others11. Impairments in these social-cognitive abilities are associated with various forms of psychopathology12.
Extract — continue reading at the source.
