Robert E. Remez
Dept. of Psychol., Barnard College, 3009 Broadway, New York, NY 10027
Descriptive studies have noted that speech perception is ineluctably multisensory if the listener can hear and see the talker. In such cases, the resolution of linguistic attributes occurs by combining auditory and visual inflow rather than by phonetic analyses in each modality before convergence. Because subjects readily acquire unanticipated proficiency in haptic and auditory integration, some researchers have conceptualized speech perception as a multisensory means of detecting a talker's phonetic production, independent of memorized sensory prototypes. These findings have motivated a definition of the problem of sensory convergence in terms that are literally multisensory: How do auditory and visual and haptic afferents coalesce? Recent studies of perceptual organization in a speech mode present a contrasting view, based on a phenomenon of unimodal convergence in speech perception. In this research employing tonal analogs of speech signals, phonetic perception occurs if auditory constituents cohere. Failure of convergence in this empirical paradigm is equal to failure of phonetic perception. Together, these studies suggest that integration of sensory attributes in speech perception is equivalent across changes in domain, from uni- to multisensory, and warrants an account of convergence that is general over changes in the scale of the sensory projection. [Work supported by NIDCD.]