S-114.780 Havaitseminen ja toiminta: Report of a research study for students (Nov 2004)
Multisensory interactions are generally stronger when stimulation arises from a similar location in space. This study investigated the effect of stimulus side on a visual illusion induced by sound, first described by Shams, Kamitani and Shimojo (2002), and replicated by our research group (Andersen, Tiippana & Sams, 2004). The original illusion is that a single flash is often perceived as two when two rapid beeps are presented concurrently with it.
In the current study, stimuli were presented either in congruent combinations so that both auditory and visual stimuli were on the same side, or in incongruent combinations so that auditory and visual stimuli were presented on opposite sides of fixation.
The study consisted of two experiments. In the constant-side experiment, stimuli were presented so that within a measurement block, the side of both auditory and visual stimuli remained the same. In the random-side experiment, stimuli were presented in a random order so the side of auditory and visual stimuli could be the same or different in each trial.
The hypotheses were:
The methods were otherwise as in Andersen, Tiippana & Sams (2004) except that the stimuli were presented to the left or right of fixation. Unimodal visual stimuli were 1, 2 or 3 flashes. A flash was a white disk with luminance 148 cd/m2 on a black background with luminance 2 cd/m2. The duration of a flash was 17 ms. Radius of the disk was 2° and the centre of the disk was 5° to the left or right of a fixation cross. Unimodal auditory stimuli were 1, 2 or 3 beeps. A beep was a Hamming-windowed sine-wave tone with a frequency of 3.5 kHz, duration of 7 ms, and sound level of 80 dB(A). The beeps were delivered through a loudspeaker either on the left or right side the fixation cross. The loudspeakers were placed on both sides of the computer screen, with the centres 15° from the fixation cross. The audiovisual stimuli were the 9 combinations of the 3 auditory and the 3 visual stimuli. Stimulus onset asynchrony was 67 ms for both auditory and visual stimuli.
The constant-side experiment consisted of four blocks, presented in counterbalanced order to the subjects. In two congruent blocks, both beeps and flashes were presented on the same side, either left or right. In two incongruent blocks, the beeps were presented on a different side than the flashes, either flashes on the left and beeps on the right or flashes on the right and beeps on the left.
In the random-side experiment, all visual and audiovisual stimuli were presented in a random order from the left or right. Two blocks with a possibility for a break in the middle were used to give the subjects rest periods.
Furthermore, two auditory blocks were included: one where subjects counted the beeps, and another where they reported whether the beeps came from the left or right.
Subjects (n=10 in each experiment) were instructed to count the flashes as accurately as possible, and to ignore any beeps. They were not informed about any other details of the stimuli, such as that the side of auditory stimulation was varied.
This overview of the results is very preliminary since I haven't found time to elaborate - my apologies for this. If you are really interested in knowing more, contact me, and I'll give you more information when I've managed to analyse the results properly.
Only the case of the classical illusion stimulus will be dealt with here since all other conditions agreed with these results.
Unimodal auditory stimuli were almost perfectly perceived, with 99.7 % correct left-right discrimination, and when counting 2 beeps, an average of 1.98 beeps were reported. A single flash alone was also almost perfectly perceived, with the average response of 1.03.
In the case of the classical illusion, a single flash was presented together with two beeps. The illusion of perceiving more than one flash was present in all conditions, reflected in the average response being clearly greater than 1.03.
All illusion conditions differed significantly from the responses for a single flash alone, but did not differ between each other.
Thus, the visual illusion induced by sound was replicated in these experiments, so that more than one flash was often perceived in the presence of two beeps. Contrary to hypotheses, however, the strength of the illusion was not affected either by the congruence or by the randomness of stimulus location. Instead, the illusion was equally strong independently of whether the beeps and flashes came from the same or different sides of fixation, and whether the side of presentation was constant or random.
These results show that a visual illusion induced by sound, discovered by Shams et al. (2002), is insensitive to stimulus location. Many other audiovisual interactions are weaker when stimuli arise from different locations, such as enhancement of visual detection by sound (Frassinetti, Bolognini & Làdavas, 2002; McDonald, Teder-Sälejärvi & Hillyard, 2000). However, this is not always the case. For example, perception of audiovisual speech is not affected by whether the voice is presented from the same location as the face (Jones & Munhall, 1997). The similarity between audiovisual speech and the 'Shams' illusion is that both are examples of categorical object perception. The current results thus suggest that audiovisual object perception is much less sensitive to stimulus location than many other audiovisual tasks based on continuous perception of stimulus qualities.
Andersen T. S., Tiippana K. and Sams M. (2004). Factors influencing audiovisual fission and fusion illusions. Cognitive Brain Research, 21, 301-308.
Frassinetti F., Bolognini N. and Làdavas E. (2002). Enhancement of visual perception by crossmodal visuo-auditory interaction. Experimental Brain Research, 147, 332-343.
Jones J. A. and Munhall K. G. (1997). The effects of separating auditory and visual sources on audiovisual integration of speech. Canadian Acoustics, 25, 13-19.
McDonald J. J., Teder-Sälejärvi W. A. and Hillyard S. A. (2000). Involuntary orienting to sound improves visual perception. Nature, 407, 906-908.
Melara R. D. (1989). Dimensional interaction between color and pitch. J Exp Psychol Hum Percept Perform., 15, 69-79.
Shams L., Kamitani Y. and Shimojo S. (2002). Visual illusion induced by sound. Cognitive Brain Research, 14, 147-152.