Audiovisual synchrony perception and organisation in complex sequences

Synchrony is one of the primary factors that affects whether we are likely to perceive sensation from different modalities as coming from the same source. In this thesis, we explore unisensory grouping and crossmodal synchrony perception amongst auditory and visual sequences, using stimuli that stream into foreground and background groups. Previous research has shown that the presence of visual stimuli that clearly formed either one or two groups could affect the number of perceptual groups formed in concurrent auditory stimuli (O‘Leary & Rhodes, 1984). In this thesis, we find that when there is no conflict in the number of groups but in there is in the items allocated to each group, that there is no visual influence on the way the auditory groups are organised – rather, we see a temporal ventriloquism effect that results in the groups appearing temporally aligned. This supports the notion that unimodal grouping will occur prior to intersensory pairing when there is a conflict. Participants showed a large tolerance for asynchrony in these experiments, and we note that the ability to make accurate audiovisual synchrony judgements appears to be affected by the ‗complexity‘ of the stimuli – we are much better at making judgements when matching single beeps or flashes as opposed to speech or music. In the subsequent experiments, we investigate whether the predictability of sequences affects whether participants report that auditory and visual sequences appear to be synchronous. Across the thesis, we examine the effect of pitch, temporal and spatial predictability, temporal density and virtual depth in a head-mounted display, and clearly find that when the load on the perceptual system is high because several of these factors are manipulated concurrently, participants become worse at determining asynchrony – in effect, they are much better able to tolerate higher levels of asynchrony between the auditory and visual stimuli.