Re: real-time information (Paris Smaragdis )

Subject: Re: real-time information
From:    Paris Smaragdis  <paris(at)>
Date:    Mon, 5 Mar 2001 16:54:02 -0500

> A stingy engineer would not decide to hugely waste valuable bits for > opening an additional frequency-dimension as does cochlea. He would by no > means carelessly loose half of information. He would follow his intuition > within the horizon of his mathematical gospels and fail to come close to > the admirable natural solution. Actually, there was no reason for the > evolution to use Fourier transform. We only may realize that the solution > is similar to this mathematical tool to a certain extent. The phase > spectrum is not included. Stingy engineering might not grasp these concepts, but what would you expect from people that live to design $5 radios? :-) The reason why the Fourier transform exists, and why any scientist, as well as our ear, would waste the extra bandwidth is one that people seldom bother to examine. Thinking that the Fourier transform is just a collection of complex sinusoids that do harmonic decomposition is the wrong approach (albeit convenient). The Fourier transform, as well as the cosine and sine families of transforms, are optimal 2nd order decompositions of markov processes. That means that if you have time series with any temporal coherence (that would be any sound), and you wanted to find a set of basis functions that would decorrelate them, you would end up with a harmonic transform (and the virtues of decorrelation and independence are well understood even by stingy engineers). Harmonic transforms are not a convenient tool we came up with one day, they are formed by the nature of coherent time series. Likewise, applying decompositions with other independence criteria you can derive constant-Q-ish, and even cochlea-lookalike transforms. Why my sudden outbreak of math trivia in an auditory list, you ask? Well, if all these harmonic transforms have been derived by observation of coherent time series, would it be far-fetched to assume the same for our perceptual preprocessors? This reasoning is right on track with the Barlowian view on redundancy reduction and perception. Regardless of how our ears are right now, there was a point in time they did not exist. But they had to grow to a form efficient enough to facilitate scene analysis, by adaptation to their stimuli. In doing so they could have strived for a form that resulted in a sparse and informative decomposition. In vision there has been a recent track of very well received papers supporting this. The same processes, when used in 2D, result in retinal and V1-like receptive fields. There are very interesting 'coincidences' and very strong theories forming in the field of mathematical modelling of perception, and all of us in this field would appreciate less snubbing. Don't blame math for being poor and unnatural. It is only a tool, and when used correctly it is a lot more interesting than measuring cochlear responses. Paris

This message came from the mail archive
maintained by:
DAn Ellis <>
Electrical Engineering Dept., Columbia University