Subject: summary of missing fundamental responses From: Petr Janata <janata(at)CAJAL.UOREGON.EDU> Date: Wed, 20 Oct 1993 13:50:44 -0700Dear Auditory list members: Thanks for your help in illuminating the body of literature relating to the neurophysiology of the missing fundamental. It was larger than I expected! Several people requested that I post a summary of the responses I received, and although the bulk of the references were posted to the entire list, I have compiled and attached below all of the responses I received that mentioned specific research. Petr ------------------------------------------------------------------------- From: Dirk Kautz <kautz(at)tyto.mpib-tuebingen.mpg.de> 1. Tomlinson RW; Schwarz DW. Perception of the missing fundamental in nonhuman primates. Journal of the Acoustical Society of America, 1988 Aug, 84(2):560-5. (UI: 89009246) Abstract: In preparation for neurophysiological experiments aimed at mechanisms of pitch perception, four rhesus monkeys were trained to press a button when the fundamental frequencies (missing or present) of two complex tones in a tone pair matched. Both tones were based on a five-component harmonic series. Zero to three of the lowest components could be missing in the first tone, while the second (comparison) tone contained all five harmonics. The range of fundamentals tested varied from 200 to 600 Hz. Three monkeys learned to match tones missing their fundamentals to comparison harmonic complexes with the same pitch, whereas the fourth monkey required the physical presence of the fundamental. Consideration of several cues available to the monkeys suggests that the animals could perceive the missing fundamental. ------------------------------------------------------------------------- From: Larry Feth <feth(at)shs.ohio-state.edu> What do you mean by "neurophysiological"? More than 10 years ago a student of mine conducted a comparison of pitch shift of the "missing" fundamental with the frequency following potential, because Steve Greenburg had said that the FFP was the neurological sign of the "pitch of the missing fundamental" I can get you the reference (its R. Chambers et al in J Acoust Soc Amer from about 1983) ------------------------------------------------------------------------ From: lwyse(at)PARK.BU.EDU The following, rather obscure article reports that they have evidence that the tonotopy in the cortex is pitch, not frequency based. (at)article{Pantevetal, author = {C. Pantev and M. Hoke and B. L{\"u}tkenh{\"o}ner and K. Lehnertz}, journal = {Science}, pages = {486-488}, title = {Tonotopic organization of the auditory cortex: {P}itch Versus Frequency Representation}, volume = {246}, year = {1989}, keywords = "pitch tonotopy in auditory cortex" } Langnor and Schreiner's work on the Inferior Colliculus all addresses the issue of the "missing fundamental". See also, Pantev for recent cortical work regarding pitch. ------------------------------------------------------------------ From: Alain de Cheveigne <alain(at)linguist.jussieu.fr> Specifically concerning the "missing fundamental", you should look at Evans (1978, 1983, 1986) and his discussion with Whitfield (1978, 1979, 1980), see also Moore (1980). Widening the topic a bit, "missing fundamental phenomena" can be explained if we suppose that the auditory system derives pitch from the period of discharge of auditory-nerve fibers that do not resolve individual partials (this period is the inverse of the missing fundamental), and/or from the shortest common super-period of fibers that do resolve partials (this common super-period is also the inverse of the missing fundamental). This idea seems to have been first proposed by Fletcher (1929, quoted by Schubert 1978). Licklider (1956, 1959, 1962) proposed a neural processing model to explain how this information could be extracted from the auditory nerve. Others have elaborated this model (Lyon 1984, 1986; Moore, 1982; van Noorden 1982; de Cheveigne, 1986; Lazzaro and Mead, 1989; Slaney 1990; Meddis and Hewitt 1988, 1991). Evidence for synchrony of discharge, needed by such mechanisms, can be found in early papers by Rose et al. (1967, 1968, 1969) and Kiang (1965). See Johnson (1980) for the frequency limits of synchrony. See Lagner (1981, 1983, 1992), Lagner and Schreiner (1988) and Schreiner and Lagner (1988) for evidence that might be interpreted as supporting Licklider's model. There's also lots of evidence for the related Jeffress model of binaural processing. Speech has rather little energy at the fundamental. In that sense, the physiological representation of speech F0 is relevant to your issue. See Delgutte (1984), Delgutte and Kiang (1984), Javel (1980), Kim and Leonard (1988), Miller and Sachs (1984), Young and Sachs (1979), Palmer et al. (1986). You might also look into work on the representation of the F0s of mixed vowels (Palmer 1988, 1990, 1992; de Cheveigne 1993; Meddis and Hewitt 1992) All this is rather biased in favor of time-domain processing and the Licklider model. Maybe someone else could provide more references in favor of other models? I hope this helps, Alain. --- de Cheveigne, A. (1986). "A pitch perception model", Proc. IEEE ICASSP, pp. 897-900. de Cheveigne, A. (1993). "Separation of concurrent harmonic sounds: Fundamental frequency estimation and a time-domain cancellation model of auditory processing.," JASA 93, pp. 3271-3290. Delgutte, B. (1984). "Speech coding in the auditory nerve: II. Processing schemes for vowel-like sounds," JASA 75, pp. 879-886. Delgutte, B. and N. Y.-S. Kiang. (1984). "Speech coding in the auditory nerve: I. Vowel-like sounds," JASA 75, pp. 866-878. Delgutte, B. and N. Y.-S. Kiang. (1984). "Speech coding in the auditory nerve: V. Vowels in background noise," JASA 75, pp. 908-918. Evans, E. F. (1986). Pitch and cochlear nerve fiber discharge patterns. In B. C. J. Moore, & R. D. Patterson (Ed.), Auditory frequency selectivity (pp. 253-264). Plenum Press. Evans, E. F. (1983). "Pitch and cochlear nerve fibre temporal discharge patterns," in Hearing-Physiological bases and psychophysics, edited by R. Klinke and R. Hartmann (Springer-Verlag, Berlin), pp. 140-146. Evans, E. F. (1978). "Place and time coding of frequency in the peripheral auditory system: Some physiological pros and cons," Audiology 17, pp. 369-420. Javel, E. (1980). "Coding of AM tones in the chinchilla auditory nerve: implications for the pitch of complex tones," JASA 68, pp. 133-146. Johnson, D. H. (1980). "The relationship between spike rate and synchrony in responses of auditory-nerve fibers to single tones," JASA 68, pp. 1115-1122. Kiang, N. Y.-S. (1965). "Discharge patterns of single fibers in the cat's auditory nerve", MIT research monograph 35, pp 154. Kim, D. O. and G. Leonard. (1988). "Pitch-period following response of cat cochlear nucleus neurons to speech sounds," in Basic issues in hearing, edited by H. Duifhuis, J. W. Horst and H. P. Wit (Academic Press, London), pp. 252-260. Langner, G. (1981). "Neuronal mechanisms for pitch analysis in the time domain," Exp. Brain Res. 44, pp. 450-454. Langner, G. (1983). "Evidence for neuronal periodicity detection in the auditory system of the guinea fowl: implications for pitch analysis in the time domain," Exp. Brain Res. 52, pp. 333-355. Langner, G. (1983). "Neuronal mechanisms for a periodicity analysis in the time domain," in Hearing-Physiological bases and psychophysics, edited by R. Klinke and R. Hartmann (Springer-Verlag, Berlin), pp. 334-341. Langner, G. (1992). "Periodicity coding in the auditory system," Hearing Research 60, pp. 115-142. Langner, G. and C. E. Schreiner. (1988). "Periodicity coding in the inferior colliculus of the cat. I. Neuronal mechanisms," J. Neurophysiol. 60, pp. 1799-1822. Lazzaro, J. and C. Mead. (1989). "Silicon modeling of pitch perception," Proc. Natl. Acad. Sci. USA 86, pp. 9597-9601. Licklider, J. C. R. (1956). "Auditory frequency analysis," in Information theory, edited by C. Cherry (Butterworth, London), pp. 253-268. Licklider, J. C. R. (1959). "Three auditory theories," in Psychology, a study of a science, edited by S. Koch (McGraw-Hill, New York), pp. 41-144. Licklider, J. C. R. (1962). "Periodicity pitch and related auditory process models," International Audiology 1, pp. 11-36. Lyon , R. F. and L. Dyer. (1986). "Experiments with a computational model of the cochlea", Proc. IEEE ICASSP, pp. Lyon, R. (1984). "Computational models of neural auditory processing", IEEE ICASSP, pp. 36.1.(1-4). Meddis, R. and M. Hewitt. (1988). "A computational model of low pitch judgement," in Basic issues in hearing, edited by H. Duifuis, J. W. Horst and H. P. Witt (Academic, London), pp. 148-153. Meddis, R. and M. J. Hewitt. (1991). "Virtual pitch and phase sensitivity of a computer model of the auditory periphery. I: pitch identification," JASA 89, pp. 2866-2882. Meddis, R. and M. J. Hewitt. (1991). "Virtual pitch and phase sensitivity of a computer model of the auditory periphery. II: phase sensitivity," JASA 89, pp. 2883-2894. Meddis, R. and M. J. Hewitt. (1992). "Modeling the identification of concurrent vowels with different fundamental frequencies," JASA 91, pp. 233-245. Miller, M. I. and M. B. Sachs. (1984). "Representation of voice pitch in discharge patterns of auditory-nerve fibers," Hearing Research 14, pp. 257-279. Moore, B. C. J. (1980). "Neural interspike intervals and pitch," Audiology 19, pp. 363-365. Moore, B. C. J. (1982). An introduction to the psychology of hearing (Academic Press, London). p. Moller, A. R. (1979). "Coding of complex sounds in the auditory nervous system," in Hearing mechanisms and speech, edited by O. Creutzfeldt, H. Scheich and C. Schzeiner (Springer-Verlag, Berlin), pp. 45-54. van Noorden, L. (1982). "Two channel pitch perception," in Music, mind, and brain, edited by M. Clynes (Plenum press, London), pp. 251-269. Palmer, A. R. (1988). "The representation of concurrent vowels in the temporal discharge patterns of auditory nerve fibers," in Basic issues in hearing, edited by H. Duifhuis, J. W. Horst and H. P. Wit (Academic Press, London), pp. 244-251. Palmer, A. R. (1990). "The representation of the spectra and fundamental frequencies of steady-state single- and double-vowel sounds in the temporal discharge patterns of guinea pig cochlear-nerve fibers," J. Acoust. Soc. Am. 88, pp. 1412-1426. Palmer, A. R., I. M. Winter and C. J. Darwin. (1986). "The representation of steady-state vowel sounds in the temporal discharge patterns of the guinea pig cochlear nerve and primarylike cochlear nucleus neurons," JASA 79, pp. 100-113. Palmer, A. R. (1992). "Segregation of the responses to paired vowels in the auditory nerve of the guinea-pig using autocorrelation," in Audition speech and language, edited by B. Schouten (Mouton-DeGruyter, Berlin), pp. (in press). Rose, J. E., J. F. Brugge, D. J. Anderson and J. E. Hind. (1967). "Phase-locked response to low-frequency tones in single auditory nerve fibers of the squirrel monkey," J.Neurophysiol. 30, pp. 769-793. Rose, J. E., J. F. Brugge, D. J. Anderson and J. E. Hind. (1968). "Patterns of activity in single auditory nerve fibres of the squirrel monkey," in Hearing mechanisms in vertebrates, edited by A. V. S. De Reuk and J. Knight (Churchill, London), pp. 144-168. Rose, J. E., J. F. Brugge, D. J. Anderson and J. E. Hind. (1969). "Some possible neural correlates of combination tones," J. Neurophysiol. 32, pp. 402-423. Rose, J. E., L. M. Kitzes, M. M. Gibson and J. E. Hind. (1974). "Observations on phase-sensitive neurons of anteroventral cochlear nucleus of the cat: nonlinearity of cochlear output," J. Neurophysiol. 37, pp. 218-253. Schreiner, C. E. and G. Langner. (1988). "Coding of temporal patterns in the central auditory nervous system," in Auditory function - Neurobiological bases of hearing, edited by G. M. Edelman, W. E. Gall and W. M. Cowan (Wiley, New York), pp. 337-361. Schreiner, C. E. and G. Langner. (1988). "Periodicity coding in the inferior colliculus of the cat. II. Topographical organization," J. Neurophysiol. 60, pp. 1823-1840. Schubert, E. (1978). "History of research on hearing," in Handbook of perception, edited by E. C. Carterette and M. P. Friedman (Academic Press, New York), pp. 41-80. Slaney. (1990). "A perceptual pitch detector", ICASSP-90, pp. 357-360. van Noorden, L. (1982). "Two channel pitch perception," in Music, mind, and brain, edited by M. Clynes (Plenum press, London), pp. 251-269. Young, E. D. and M. B. Sachs. (1979). "Representation of steady-state vowels in the temporal aspects of the discharge patterns of populations of auditory-nerve fibers," JASA 66, pp. 1381-1403. Whitfield, I. C. (1978). "The neural code," in Handbook of perception, vol IV, edited by E. C. Carterette and M. P. Friedman (Academic Press, New York), pp. 163-183. Whitfield, I. C. (1979). "Periodicity, pulse interval and pitch," Audiology 18, pp. 507-512. Whitfield, I. C. (1980). "Theory and experiment in so-called pulse-interval pitch," Audiology 20, pp. 86-88. ------------------------------------------------------------------ From: "Robert E. Remez" <REMEZ(at)paradise.barnard.columbia.edu> Schwartz, D. W., & Tomlinson, R. W. (1990). Spectral response patterns of auditory cortex neurons to harmonic complex tones in alert monkey (Macaca mulatta). Journal of Neurophysiology, 64, 282-298. ------------------------------------------------------------------ From: parncutt(at)sound.music.mcgill.ca Yet another interesting reference is: Zatorre, R.J. (1988). Pitch perception of complex tones and human temporal-lobe function. Journal of the Acoustical Society of America, 84, 566-572. Zatorre's (1988) results suggest that Heschl's gyri and the surrounding cortex in the right hemisphere play a crucial role in extracting the pitch corresponding to the fundamental of a complex tone. His tones had fundamental frequencies in the range 200 to 1000 Hz. Some other, related papers by Zatorre and colleagues are: Samson, S., & Zatorre, R.J. (1991). Recognition memory for text and melody of songs after unilateral temporal lobe lesion: Evidence for dual encoding. JEP: Learning, Memory, and Cognition, 17, 793-804. Samson, S., & Zatorre, R.J. (1992). Learning and retention of melodic and verbal information after unilateral temporal lobectomy. Neurophsycholgia, 30 (9), 815-826. Zatorre, R.J. (1989). Intact absolute pitch ability after left temporal lobectomy. Cortex, 25, 567-580. Zatorre, R.J., Evans, A.C., Meyer, E., & Gjedde, A. (1992). Lateralization of phonetic and pitch discrimination in speech processing. Science, 256, 846-849. Zatorre, R.J., & Beckett, C. (1989). Multiple coding strategies in the retention of musical tones by possessors of absolute pitch. Memory and Cognition, 17, 582-589. Zatorre, R.J., & Samson, S. (1991). Role of the right temporal neocortex in retention of pitch in auditory short-term memory. Brain, 114, 2403-2417. ------------------------------------------------------------------ From: Alain de Cheveigne <alain(at)linguist.jussieu.fr> Brugge, J. F., N. A. Dubrowsky, M. Aitkin and D. J. Anderson. (1969). "Sensitivity of single neurons in auditory cortex of cat to binaural tonal stimulation; effects of varying time and intensity.," J. Neurophysiol. (vol ???), pp. 1005-1024. Carney, L. H. and T. C. T. Yin. (1989). "Responses of low-frequency cells in the inferior colliculus to interaural time differences of clicks: excitatory and inhibitory components," J. Neurophysiol. 62, pp. 144-161. Carr, C. E. and M. Konishi. (1990). "A circuit for detection of interaural time differences in the brain stem of the barn owl," J. Neuroscience 10, pp. 3227-3246. Chan, J. C. K., T. C. T. Yin and A. D. Musicant. (1987). "Effects of interaural time delays of noise stimuli on low-frequency cells in the cat's inferior colliculus. II. Responses to band-pass filtered noises," J. Neurophysiol. 58, pp. 543-561. Jeffress, L. A. (1948). "A place theory of sound localization," J. Comp. Physiol. Psychol. 41, pp. 35-39. Joseph, A. W. and R. L. Hyson. (1993). "Coincidence detection by binaural neurons in the chick brain stem," J. Neurophysiol. 69, pp. 1197-1211. Konishi, M., T. T. Takahashi, H. Wagner, W. E. Sullivan and C. E. Carr. (1988). "Neurophysiological and anatomical substrates of sound localization in the owl," in Auditory function - neurobiological bases of hearing, edited by G. M. Edelman, W. E. Gall and W. M. Cowan (Wiley, New York), pp. 721-745. Kuwada, S., R. Batra and T. R. Stanford. (1989). "Monaural and binaural response properties of neurons in the inferior colliculus of the rabbit: effects of sodium pentobarbital," J. Neurophysiol. 61, pp. 269-282. Kuwada, S., T. C. T. Yin, L. B. Haberly and R. E. Wickesberg. (1980). "Binaural interaction in the cat inferior colliculus: physiology and anatomy," in Psychophysical, physiological and behavioral studies in hearing, edited by G. v. d. Brink and F. A. Bilsen (Delft University Press, pp. 401-411. Palmer, A. R., A. Rees and D. Caird. (1990). "Interaural delay sensitivity to tones and broad band signals in the guinea-pig inferior colliculus," HR 50, pp. 71-86. Patterson, R. D. (1987). "A pulse-ribbon model of monaural phase perception," JASA 82, pp. 1560-1586. Patterson, R. D., K. Robinson, J. Holdsworth, D. McKeown, C. Zhang and M. Allerhand. (1992). "Complex sounds and auditory images," in Auditory physiology and perception, edited by Y. Cazals, K. Horner and L. Demany (Pergamon Press, Oxford), pp. 429-446. Suga, N. (1990). "Cortical computational maps for auditory imaging," Neural Networks 3, pp. 3-21. Yin, T. C. T. and J. C. K. Chan. (1988). "Neural mechanisms underlying interaural time sensitivity to tones and noise," in Auditory function - Neurological bases of hearing, edited by G. M. Edelman, W. E. Gall and W. M. Cowan (Wiley, New York), pp. 385-430. Yin, T. C. T. and J. C. K. Chan. (1990). "Interaural time sensitivity in medial superior olive of cat," J. Neurophysiol. 64, pp. 465-488. Yin, T. C. T., J. C. K. Chan and L. H. Carney. (1987). "Effects of interaural time delays of noise stimuli on low-frequency cells in the cat's inferior colliculus. III. Evidence for cross-correlation," J. Neurophysiol. 58, pp. 562-583. Yin, T. C. T., S. Kuwada and Y. Sujaku. (1984). "Interaural time sensitivity of high-frequency neurons in the inferior colliculus," JASA 76, pp. 1401-1410. ------------------------------------------------------------------ From: MD37000 <MD37(at)musica.mcgill.ca> I see that my colleague Richard Parncutt beat me to it in giving you the reference to my work (thank you for the publicity, Richard!) on the missing fundamental. We did find pretty clear evidence for a specific role for the right primary auditory region (and surrounding cortex) in perception of the missing fundamental. This was in accord with an old study of Whitfield (JASA, 1980, 67, 644-647), though not, of course in terms of hemispheric differences. We are now continuing and refining the work with better definition (via brain imaging) of the precise region of auditory cortex that has been damaged, for we have not been able to determine if the koniocortex itself is the most important region. We have also continued some of this research with a patient who has bilateral vascular lesions of the primary auditory cortices, to be presented at the society for Neuroscience (Bharucha, Tramo, and Zatorre, SN abstracts, 1993, p. 1687, abs. number 693.4). Two other abstracts are being presented that are also quite relevant: the first deals with neuromagnetic recordings in humans (Langner et al., SN abstracts, 1993, p.1423, number 581.11); the other deals with unit recordings in the monkey's auditory cortex (Riquimaroux et al., p. 1423, number 581.12). Hope this information was useful. Robert Zatorre Montreal Neurological Institute MD37(at)MUSICA.MCGILL.CA ------------------------------------------------------------------ From: Dick_Lyon(at)hypermail.apple.com The Delgutte and Cariani reference Alain mentioined is Belgutte B, Cariani P (1992): "Coding of pitch of harmonic and inharmonic complex tones in the interspike intervals of auditory-nerve fibers", in The Processing of Speech: Schouten M (ed.), Mouton-De Gruyter, Berlin. (there may be others, but this one is the only one I can find immediately). Alain, thanks for the excellent bibliographies. By the way, it's time we stop thinking of "missing fundamental" as a "problem". For Helmholtz maybe it was a problem, but no more. Dick Lyon ATG/Interactive Media/Perception Systems Apple Computer ms 301-3M One Infinite Loop Cupertino, CA 95014 (408) 974-4245