The interplay of within and between-sensory binding cues in audiovisual scene analysis

Poster Session A - Saturday, March 29, 2025, 3:00 – 5:00 pm EDT, Back Bay Ballroom/Republic Ballroom

Qiong Wu¹, Noppeney Uta¹; ¹Donders Institute for Brain, Cognition, and Behaviour

To transform the sensory cacophony into a coherent percept of the environment, the brain should integrate signals from common causes and segregate those from separate causes. How the brain combines within- and between-sensory binding cues for causal inference in complex audiovisual scenes remains unknown. We addressed this using novel ambiguous audiovisual scenes in which human observers (N=24) were presented with alternating high- and low-pitched tones from the center, alone or synchronized with blobs flashing alternately in the left and right hemifields. Across auditory trials the auditory sequences were presented at three levels of frequency separation. Across audiovisual (AV) trials the flashes were presented at three levels of symmetric eccentricities, which manipulated both the inter-flash distance and the audiovisual spatial disparity. By combining the subjective causal inference reports (“one” vs. “two streams”) and objective deviant detection performance (d-prime), our results confirm previous findings that larger frequency separation enhances auditory stream segregation and thereby facilitates deviant detection. Crucially, inter-flash distance, a visual binding cue, influenced auditory perception non-monotonically: At small distances, close flashes were integrated into one stream of apparent motion flashes, promoting auditory stream integration via AV binding. At intermediate distances, the perception of two distinct visual streams enhanced auditory stream segregation via AV binding. At large distances, auditory stream segregation was comparable to that in a purely auditory context, as pronounced AV spatial disparities hindered AV binding. Our results show that the brain combines within- and between-sensory binding cues to attribute sensory signals to their underlying causes.

Topic Area: PERCEPTION & ACTION: Multisensory