Visual semantic encoding and identification of naturalistic movies via high-density diffuse optical tomography

Understanding how the brain represents meaning in real-world contexts is essential for fundamental neuroscience and clinical applications. Existing approaches rely on constrained scanning environments or conventional functional near-infrared spectroscopy (fNIRS), which is limited to sparse sampling or block-design paradigms. Here we show that high-density diffuse optical tomography (HD-DOT) supports naturalistic semantic encoding and decoding.

We collect 3.5 hours of naturalistic movie data from six participants using stimuli labeled with 1708 categories. Encoding models predict voxel-level responses, yielding single semantic category maps consistent with prior fMRI studies. DOT responses capture sufficient semantic content to identify which clips participants viewed.

A shared low-dimensional semantic space captures common semantic structure, and clustering analyses reveal interpretable higher-order dimensions such as animate and social agents. These findings show that DOT recovers distributed, high-dimensional semantic representations from naturalistic movies, opening new directions for naturalistic real-world neuroimaging in populations and scenarios difficult to access with fMRI. The authors thank all participants for their generous time dedicated to this research.

This work was funded by the Bill and Melinda Gates Foundation [Grant No. OPP1184813 awarded to J.P.C], the National Institutes of Health [grant numbers U01EB027005, R01NS090874, R01EB03491902 awarded to J.P.C., grant number F32DC022178 awarded to J.T., and grant number F31NS110261 awarded to Z.E.M.], the Imaging Sciences Pathway Fellowship awarded to W.F. and M.F. [grant number T32EB014855], and the SPIE-Franz Hillenkamp Postdoctoral Fellowship awarded to M.F. These authors jointly supervised this work: Alexander G.

Imaging Science, Washington University in St. Louis, MO, USA Wiete Fehner, Aahana Bajracharya & Joseph P. Culver Mallinckrodt Institute of Radiology, Washington University in St.

Louis School of Medicine, St. Louis, MO, USA Wiete Fehner, Morgan Fogarty, Dana Wilhelm, Aahana Bajracharya, Zachary E. Culver Department of Speech, Language, and Hearing Sciences, The University of Texas at Austin, Austin, TX, USA Child Study Center, School of Medicine, Yale University, New Haven, CT, USA Department of Biomedical Engineering, Washington University in St.