The impact of a synchronized multimodal AI interface configuration on consumer experience journey and satisfaction

With the rapid integration of multimodal artificial intelligence (AI) into digital interfaces, understanding how combined visual, auditory, and textual cues influence consumers' experience journeys and satisfaction has become critically important for both scholars and practitioners. While prior research has examined individual modalities in isolation, the comparative effects of a specific rich multimodal interface configuration versus a minimal-modality baseline across different stages of the consumer journey remain underexplored. This study draws on cue utilization theory, presence theory, and the customer journey framework to investigate how exposure to a synchronized multimodal AI interface (text + voice + animated avatar) versus a text-only interface affects consumer satisfaction through three experimentally induced journey stages: search, evaluation, and post-purchase.

A between-subjects online experiment (N = 320 recruited; 312 retained after screening) was conducted using a simulated e-commerce platform where participants interacted with an AI shopping assistant featuring either text-only or synchronized text-voice-avatar cues. Results from SEM indicate that the synchronized multimodal interface was associated with higher perceived diagnosticity and immersion, which in turn were related to higher satisfaction across all three stages. The configuration–satisfaction contrast was largest for evaluation satisfaction (Cohen’s d = 1.58, 95% CI [1.33, 1.83]); the indirect-effect patterns via perceived diagnosticity and immersion were statistically consistent with, but not demonstrative of, the proposed pathways.

We emphasize that the two conditions differ not only in synchronization but also in modality count, anthropomorphic cues, and social presence; accordingly, all findings are interpreted as pertaining to the interface configuration as a whole, not to integration quality as an isolated construct. Theoretical contributions include extending cue utilization theory in the form of a speculative conjecture and identifying stage-specific patterns. Managerial implications suggest that e-commerce platforms should consider synchronized multimodal cue design, particularly during product comparison phases in simulated and early deployment contexts, and should be regarded as preliminary pending field validation.

I would like to express my sincere gratitude to all those who have contributed to this study with their support and assistance. The authors did not receive support from any organization for the submitted work. School of Business Administration, Zhongnan University of Economics and Law, Wuhan, 430073, China The authors have declared that no competing interests exist.

All procedures complied with the Declaration of Helsinki and relevant ethical guidelines. This non-invasive anonymous online behavioral experiment was granted an ethical exemption by the Institutional Review Board/Ethics Committee of Zhongnan University of Economics and Law. Informed consent was obtained from all participants via Prolific Academic's standard user agreement, and no identifiable personal information was collected.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Below is the link to the electronic supplementary material. Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material.