Facial impressions are central to social interaction, shaping judgments of attractiveness, dominance, and trustworthiness. Although psychophysical research has identified factors that contribute to these impressions, computational methods, particularly deep learning, offer a complementary, data-driven identification of impression-relevant facial cues. We trained a Visual Geometry Group 19-layer (VGG19) regression model to predict impression ratings from facial images and used explainable AI methods to interpret its predictions.
We generated Grad-CAM and Guided Grad-CAM visualizations to identify decision-relevant facial regions at coarse and fine-grained levels. We used a large language model to produce structured, reproducible linguistic summaries of Guided Grad-CAM outputs and quantitatively cross-checked these summaries using an independent coarse region-of-interest (ROI) analysis that partitioned heatmap activation across the eyes, nose, mouth, and the remaining facial area. The results indicate impression-specific activation patterns.
Attractiveness showed distributed emphasis across central facial regions, dominance involved balanced contributions with stable upper-face cues, and trustworthiness was characterized by strong mouth- and eye-related activation. The language-based summaries were repeatedly consistent with ROI-level distributions, supporting their use as constrained descriptions of salient activation patterns. This study demonstrated a practical workflow that integrated visualization, structured language outputs, and quantitative cross-checking to improve interpretability in facial impression prediction models.
We thank Editage (www.editage.com) for English language editing. This study was supported by JSPS KAKENHI Grant Number 25KJ0044 awarded to T.S. and Grant Numbers 24H02199 and 24K21502 awarded to H.K. Graduate School of Engineering, The University of Tokyo, 3-1-7 Hongo, Bunkyo-ku, Tokyo, 113-8654, Japan Graduate School of Human Relations, Keio University, 2-15-45 Mita, Minato-ku, Tokyo, 108-8345, Japan Faculty of Letters, Keio University, 2-15-45 Mita, Minato-ku, Tokyo, 108-8345, Japan Correspondence to Takanori Sano or Hideaki Kawabata.
The authors declare no competing interests. Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material.
You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Extract — continue reading at the source.
